Data Engineer

Key Responsibilities

• Design, develop, and maintain scalable data pipelines and ETL workflows.
• Build and optimize data processing solutions using Python, PySpark, and Databricks.
• Develop and maintain Delta Tables and work with Parquet file formats.
• Implement workflow orchestration using Apache Airflow or ADF
• Perform data transformation, validation, and optimization activities.
• Collaborate with cross-functional teams to understand business requirements and deliver data solutions.
• Optimize data pipelines for performance, scalability, and reliability.
• Implement testing frameworks and ensure data quality.
• Troubleshoot and resolve production issues.

Mandatory Skills

• Strong experience in Python development.
• Hands-on experience with Pandas and PySpark.
• Experience in data engineering and workflow optimization.
• Strong knowledge of Databricks.
• Experience with Delta Tables and Parquet.
• Good understanding of ETL and data pipeline development.