Data Engineer

Data Engineer

Role: Data Engineer
Experience: 5+ Years
Location: Hyderabad
Employment Type: Full-Time

Job Summary

We are seeking a highly skilled Data Engineer to design, build, and optimize scalable data platforms and pipelines. The ideal candidate will have strong expertise in Snowflake, AWS Data Lake (S3), Apache Iceberg, AWS Glue, DBT, Kafka, and Python, along with experience in modern cloud-based data engineering and analytics environments.

Key Responsibilities

• Design, develop, and maintain scalable data pipelines using Python, AWS Glue, and DBT.
• Build and manage enterprise data lakes on Amazon S3.
• Implement and optimize data storage solutions using Apache Iceberg tables.
• Develop and maintain data warehouse solutions in Snowflake.
• Integrate real-time and batch data processing using Apache Kafka.
• Create robust ETL/ELT frameworks for data ingestion, transformation, and reporting.
• Monitor data pipeline performance and ensure data quality, reliability, and governance.
• Collaborate with Data Scientists, Analysts, Architects, and Business Teams to deliver data solutions.
• Implement data security, access controls, and compliance standards.
• Troubleshoot and resolve production data issues while ensuring SLA adherence.

Required Skills

• Strong experience with Snowflake Data Warehouse.
• Hands-on experience with AWS S3 Data Lake architecture.
• Expertise in Apache Iceberg for large-scale analytical datasets.
• Experience in AWS Glue for ETL development and orchestration.
• Strong proficiency in DBT (Data Build Tool).
• Experience with Apache Kafka for streaming data solutions.
• Advanced programming skills in Python.
• Good understanding of SQL and performance optimization.
• Experience with data modeling, partitioning, and data governance concepts.
• Exposure to CI/CD and DevOps practices for data platforms.

Preferred Skills

• Experience with AWS services such as Lambda, Athena, Redshift, EMR, or Step Functions.
• Knowledge of Airflow or other workflow orchestration tools.
• Familiarity with Terraform and Infrastructure as Code (IaC).
• Understanding of Data Mesh and Modern Data Architecture principles.