Data Engineer

Key Responsibilities

• Develop and maintain batch and real-time data pipelines to support business requirements.
• Build and maintain ETL workflows for data ingestion, transformation, and integration.
• Work with data warehouses and data lakes to support data storage and processing requirements.
• Monitor pipeline execution, investigate data discrepancies, and resolve technical issues.
• Perform unit testing, data validation, and code debugging to ensure data accuracy and reliability.
• Collaborate with senior engineers, analysts, and business teams to deliver data engineering solutions.
• Follow established coding standards, documentation practices, and deployment procedures.

Mandatory Requirements

• 3+ years of relevant experience in Data Engineering.
• Strong hands-on proficiency in Python programming and OOP concepts.
• Good knowledge of SQL, including complex queries, joins, and query optimization.
• Practical experience with Apache Spark / PySpark for distributed data processing.
• Hands-on experience with AWS data services such as AWS Glue, Amazon S3, Lambda, Redshift, and Athena.
• Understanding of ETL/ELT processes, data warehouses, and data lakes.
• Familiarity with data quality, data validation, and data governance principles.
• Working knowledge of Git and Agile development practices.

Preferred

• Exposure to AWS EMR, Glue Data Catalog, and Lake Formation.
• Experience in BFSI, NBFC, or FinTech.
• Familiarity with JIRA, Confluence, Jenkins, or dashboard tools such as Tableau and Amazon QuickSight.