Manager

Role & responsibilities

Job Description - Data Engineer Lead

Position: Data Engineer Lead
Experience: 12-14 Years
Location: Hyderabad
Employment Type: Full Time

Job Summary

We are seeking an experienced Data Engineer Lead with strong expertise in DBT, Snowflake, AWS S3, AWS Glue, PySpark, and Modern Data Engineering practices. The ideal candidate will lead the design, development, optimization, and maintenance of scalable data pipelines and cloud-based data platforms while mentoring a team of data engineers.

Key Responsibilities

• Lead end-to-end design and development of enterprise-scale data solutions.
• Build and optimize ETL/ELT pipelines using PySpark, DBT, AWS Glue, and Snowflake.
• Design scalable data architectures on AWS cloud.
• Develop and maintain data transformation models using DBT.
• Implement data ingestion frameworks from multiple structured and unstructured sources.
• Optimize Snowflake data warehouse performance, storage, and query execution.
• Work closely with business stakeholders, architects, and analytics teams.
• Ensure data quality, governance, security, and best practices.
• Lead code reviews, technical discussions, and mentor junior team members.
• Drive automation, CI/CD, monitoring, and deployment of data pipelines.

Required Skills

Must Have

• Strong experience in Snowflake
• Hands-on expertise in DBT (Data Build Tool)
• Advanced PySpark development experience
• Strong AWS experience with:
• S3
• AWS Glue
• IAM
• Lambda (Good to Have)
• Step Functions (Good to Have)
• Strong SQL and Data Modelling skills
• ETL/ELT Pipeline Development
• Python Programming
• Performance Tuning and Troubleshooting

Good to Have

• Airflow
• Kafka
• Terraform
• Databricks
• Data Governance Tools
• CI/CD Implementation