Aws Data Engineer

Job Summary

We are seeking an experienced Senior Data Engineer to design, build, and support scalable data integration and data processing solutions across cloud-based platforms. The ideal candidate will have strong expertise in ETL/ELT development, large-scale data processing, AWS services, workflow orchestration, and modern data architecture. This role requires close collaboration with cross-functional teams to deliver reliable, high-performance data pipelines that support business intelligence, analytics, and operational data needs.

Key Responsibilities

• Design, develop, and maintain scalable ETL/ELT pipelines for data ingestion, transformation, and integration.
• Build and support data workflows across cloud-based environments and distributed data platforms.
• Develop robust data solutions capable of processing large volumes of structured and unstructured data.
• Create and optimize SQL queries, data models, and database solutions to ensure data quality and performance.
• Implement orchestration and scheduling workflows using tools such as Apache Airflow, Azure Data Factory, or AWS Step Functions.
• Design resilient and fault-tolerant data pipelines with logging, monitoring, retry mechanisms, idempotency, and recovery capabilities.
• Work with data lakes, data warehouses, and relational/non-relational databases to support enterprise analytics.
• Integrate APIs, streaming platforms, and event-driven architectures into data engineering solutions.
• Collaborate with architects, software engineers, analysts, and business stakeholders to define and deliver data requirements.
• Support DevOps practices including CI/CD pipelines, automated testing, containerization, and infrastructure-as-code.
• Participate in code reviews, troubleshooting, performance tuning, and production support activities.
• Ensure adherence to data governance, security, compliance, and quality standards.

Required Qualifications

• Bachelor's degree in Computer Science, Engineering, Information Systems, or a related technical discipline. Equivalent practical experience may be considered.
• 5+ years of experience in Data Engineering, ETL/ELT development, or Data Integration.
• Strong experience designing and supporting large-scale data processing solutions in cloud or distributed environments.
• Advanced proficiency in SQL.
• Hands-on experience in at least one programming language:
• Python
• Java
• Scala
• Spark SQL
• Experience with workflow orchestration tools such as:
• Apache Airflow
• Azure Data Factory (ADF)
• AWS Step Functions
• Strong understanding of:
• Relational databases
• NoSQL databases
• Data warehouses
• Data lake architectures
• Experience implementing monitoring, logging, error handling, retry logic, and recovery processes for production data pipelines.
• Excellent analytical, problem-solving, communication, and collaboration skills.

Preferred Qualifications

• Experience working with AWS cloud data platforms.
• Hands-on experience with:
• AWS Lambda
• Apache Kafka
• Knowledge of:
• Data Modeling
• Metadata Management
• Data Governance
• Master Data Management (MDM)
• Experience with:
• Docker
• Kubernetes
• CI/CD pipelines
• Infrastructure as Code (Terraform/CloudFormation)
• Experience working in regulated industries such as:
• Financial Services
• Banking
• Healthcare
• Insurance

Technical Environment

The successful candidate will work within a modern data engineering ecosystem that may include:

• AWS Cloud Services
• Data Lakes and Data Warehouses
• Distributed Compute Platforms
• Apache Kafka and Streaming Systems
• APIs and Data Integration Services
• Apache Airflow / AWS Step Functions
• Git and Source Control Management
• Automated Testing Frameworks
• Monitoring and Observability Platforms
• Docker and Containerized Deployments
• CI/CD and Infrastructure as Code Tools