AWS Data Engineer (Blr/Chn/Hyd/Kochi/Kol/Pune)

We are seeking a highly skilled AWS Data Engineer with 6+ years of experience in designing, developing, and optimizing scalable cloud-based data platforms. The ideal candidate should have strong expertise in AWS services, PySpark, Python, SQL, ETL/ELT development, and large-scale data processing. The candidate will work closely with business stakeholders, data architects, and analytics teams to build robust and efficient data solutions.

Key Responsibilities

• Design, develop, and maintain scalable data pipelines on AWS.
• Build and optimize ETL/ELT workflows using PySpark, AWS Glue, and EMR.
• Develop data ingestion frameworks for structured and unstructured data.
• Design and implement Data Lake and Data Warehouse solutions.
• Process large-scale datasets using Spark and distributed computing techniques.
• Create and maintain data models supporting reporting and analytics.
• Ensure data quality, governance, security, and compliance standards.
• Monitor, troubleshoot, and optimize production data pipelines.
• Implement CI/CD practices for data engineering solutions.
• Collaborate with cross-functional teams in Agile environments.

Mandatory Technical Skills AWS Services

• Amazon S3
• AWS Glue
• EMR
• Lambda
• Athena
• Redshift
• CloudWatch
• IAM
• Step Functions

Data Engineering

• Data Lakes
• Data Warehousing
• ETL / ELT Development
• Batch Processing
• Data Modeling
• Data Migration

Big Data Technologies

• Apache Spark
• PySpark
• Spark SQL

Programming

• Python
• SQL
• Shell Scripting

DevOps & Automation

• Git
• CI/CD Pipelines
• Infrastructure as Code (Terraform preferred)

Good-to-Have Skills

• Databricks
• Snowflake
• Kafka
• Airflow
• Hadoop Ecosystem
• Docker
• Kubernetes
• Delta Lake
• Streaming Data Pipelines
• BFSI Domain Experience

Desired Competencies Technical

• Distributed Data Processing
• Performance Tuning
• Cloud Architecture
• Data Security & Governance
• Data Integration Patterns