Aws Data Engineer (6 years To 12 years)

Key Responsibilities:

• Design, develop, and maintain scalable data pipelines on AWS.
• Build and optimize ETL/ELT workflows using PySpark, AWS Glue, and EMR.
• Develop data ingestion frameworks for structured and unstructured data.
• Design and implement Data Lake and Data Warehouse solutions.
• Process large-scale datasets using Spark and distributed computing techniques.
• Create and maintain data models supporting reporting and analytics.
• Ensure data quality, governance, security, and compliance standards.
• Monitor, troubleshoot, and optimize production data pipelines.
• Implement CI/CD practices for data engineering solutions.
• Collaborate with cross-functional teams in Agile environments.

Mandatory Technical Skills:

1) AWS Services: Amazon S3, AWS Glue, EMR, Lambda, Athena, Redshift, CloudWatch, IAM, Step Functions

2) Data Engineering: Data Lakes, Data Warehousing, ETL / ELT Development, Batch Processing, Data Modeling, Data Migration

3) Big Data Technologies: Apache Spark, PySpark, Spark SQL

4) Programming: Python, SQL, Shell Scripting

5) DevOps & Automation: Git, CI/CD Pipelines, Infrastructure as Code (Terraform preferred)

Good-to-Have Skills:

• Databricks
• Snowflake
• Kafka
• Airflow
• Hadoop Ecosystem
• Docker
• Kubernetes
• Delta Lake
• Streaming Data Pipelines
• BFSI Domain Experience

Desired Technical Competencies:

• Distributed Data Processing
• Performance Tuning
• Cloud Architecture
• Data Security & Governance
• Data Integration Patterns