Data Engineer

Job Title: Data Engineer Databricks, Python & ADF

Experience: 5– 9Years
Location: Bangalore
Notice Period: Immediate to 15 Days.

Job Summary

We are looking for an experienced Data Engineer with strong hands-on expertise in Azure Databricks, Python, PySpark, SQL, and Azure Data Factory (ADF). The candidate will be responsible for designing, developing, and maintaining scalable data pipelines and ETL processes to support enterprise data engineering and analytics requirements.

Key Responsibilities

Design, develop, and maintain scalable data pipelines using Azure Databricks and Azure Data Factory.
Develop data transformation and processing solutions using Python and PySpark.
Build and optimize ETL/ELT workflows for processing large volumes of data.
Write complex SQL queries and optimize database performance.
Integrate data from multiple sources into data lakes and data warehouses.
Monitor pipeline performance, troubleshoot failures, and ensure data quality.
Collaborate with data engineers, architects, and business stakeholders.
Follow best practices for data security, performance optimization, and pipeline reliability.

Required Technical Skills

Azure Databricks: Hands-on experience developing data engineering solutions.
Python: Strong programming and data processing skills.
PySpark: Experience with distributed data processing and transformations.
Azure Data Factory (ADF): Experience creating pipelines, datasets, linked services, and triggers.
SQL: Strong querying, joins, aggregations, and query optimization.
ETL/ELT: Experience building and maintaining data integration workflows.
Azure Data Lake Storage (ADLS): Experience working with cloud-based data storage.

Preferred Skills

Experience with Azure Synapse Analytics.
Knowledge of Delta Lake and lakehouse architecture.
Familiarity with Git, CI/CD, and Azure DevOps.
Understanding of data warehousing concepts and dimensional modelling.

Candidate Profile

Bachelor’s or Master’s degree in Computer Science, Information Technology, or a related field.
Strong problem-solving and analytical skills.
Ability to work independently and collaborate with cross-functional teams.
Practical experience delivering production-grade data pipelines.