Azure Data Engineer

Job Description Data Engineer

Position: Data Engineer

Experience: 3+ Years

Location: Bangalore

Key Responsibilities

• Design, develop, and maintain scalable data pipelines using PySpark and Databricks.
• Build and optimize ETL/ELT workflows for processing large volumes of structured and unstructured data.
• Develop and manage data integration solutions using Azure Data Factory (ADF).
• Collaborate with data analysts, business stakeholders, and cross-functional teams to understand data requirements.
• Optimize Spark jobs for performance, scalability, and cost efficiency.
• Ensure data quality, governance, and security across data platforms.
• Troubleshoot and resolve data pipeline issues while ensuring high availability.

Required Skills

• Strong hands-on experience with PySpark.
• Experience working with Databricks for data engineering and analytics.
• Hands-on experience with Azure Data Factory (ADF).
• Strong knowledge of SQL and data warehousing concepts.
• Good understanding of Azure cloud services and ETL processes.
• Experience with performance tuning and optimization of Spark jobs.
• Excellent analytical, problem-solving, and communication skills.

• Data Warehousing concepts (Star Schema, Snowflake Schema, ETL/ELT, Dimensional Modelling)
• Experience in designing and developing scalable data pipelines
• Strong understanding of data integration, transformation, and optimization techniques

Good to Have

• Experience with Power BI for reporting and dashboard development.
• Knowledge of Azure Data Lake, Delta Lake, or Synapse Analytics.
• Familiarity with CI/CD pipelines and version control tools such as Git.

Education

• Bachelor's or Master's degree in Computer Science, Information Technology, or a related field.