Azure Data Engineer

Hyderabad

Datbricks,CI/CD,pyspark,devops, Data Platform

Employment Type: Full Time
Education Qualification: Bachelor's degree in Computer Science or related field or higher with minimum 3 years of relevant experience.
Position Description: Design, develop, and maintain scalable ETL/ELT pipelines using Databricks and PySpark.
Develop data processing solutions using Python and Spark for large-scale data transformation.
Write complex SQL queries for data extraction, transformation, validation, and reporting.
Build and optimize data workflows to improve performance, scalability, and reliability.
Work with structured and semi-structured data from multiple data sources.
Collaborate with data analysts, business stakeholders, and development teams to understand data requirements.
Perform data quality checks, troubleshooting, and root cause analysis for data issues.
Optimize Spark jobs and SQL queries for improved performance.
Follow coding standards, documentation, and best practices for data engineering.
Participate in code reviews and support production deployments.

Responsibilities:
Minimum 3 years of experience in Data Engineering.
Strong hands-on experience with Databricks.
Good experience in PySpark and Apache Spark.
Strong programming skills in Python.
Proficiency in SQL, including complex queries, joins, window functions, and performance tuning.
Experience in developing ETL/ELT pipelines.
Knowledge of data warehousing concepts and data modeling.
Experience with Git or other version control systems.
Strong analytical, problem-solving, and debugging skills.



Must-Have Skills:
Strong experience Databricks
Hands-on experience Data & Delta Lake
Experience with PySpark and Apache Spark.
Experience with Python.
Experience with ETL/ELT pipelines.
Excellent communication and leadership skills

Good-to-Have Skills:
data warehousing concepts
Experience with Git or other version control systems.
Background in software development or system architecture