Azure Data Engineer

Title: Azure Data Engineer

Location: remote India

Industry: Pharmaceutical

Type: till end of the year, strong possibility of extension of going well

Rate: Negotiable /hr USD

Description: Job Description: Azure Data Engineer

How You'll Spend Your Day

• Develop and support data engineering solutions using Azure Data Factory and Azure Databricks.

• Build, enhance, and maintain data pipelines for ingestion, transformation, validation, and delivery.

• Work on hands-on development activities across data acquisition, processing, and consumption layers.

• Analyze functional specifications and user stories, and translate them into technical implementation tasks.

• Develop scalable and reusable data processing logic using PySpark, SQL, and Databricks notebooks.

• Create, schedule, monitor, and troubleshoot ADF pipelines, triggers, linked services, datasets, and integration runtimes.

• Support data integration from multiple source systems into the data lake or data warehouse environment.

• Perform unit testing, data validation, debugging, and defect resolution for data pipelines and transformation logic.

• Monitor pipeline execution, investigate failures, and implement fixes to improve reliability and performance.

• Participate in code reviews and follow agreed development standards, naming conventions, and best practices.

• Work with business analysts, senior engineers, architects, and other team members to clarify requirements and deliver assigned development tasks.

• Support automation and process improvement activities to improve data processing efficiency and accuracy.

• Maintain technical documentation for pipelines, notebooks, data flows, and operational procedures.

• Provide status updates on assigned tasks, highlight issues or blockers, and support timely resolution.

• Work independently on assigned development activities while collaborating effectively within the team.

• Stay willing and enthusiastic to learn new data engineering technologies, tools, and development practices.

Stay current with Azure data engineering technologies and continuously learn new tools, frameworks, and best practices.

Your Qualification and Experience

• Bachelor's degree in computer science or 8+ years relevant experience

• Minimum 3 years of experience as a Developer; minimum 4 years of experience as a Senior Developer

• Minimum 3 years of experience developing SQL. DataBricks/DataLke experience required

• Minimum 3 years of experience in working with cross-functional teams preferred

Skills and Capabilities

• Strong hands-on development experience with Azure Data Factory (ADF) and Azure Databricks (ADB).

• Proficiency in Python, PySpark, Spark SQL, and SQL for data engineering development.

• Experience developing, enhancing, and supporting enterprise-scale ETL/ELT data pipelines.

• Good understanding of Azure data services, including ADLS Gen2, Azure Key Vault, Azure DevOps, and related cloud components.

• Knowledge of Delta Lake, Medallion Architecture, and Data Lake concepts.

• Understanding of data modeling, data quality, and data governance principles.

• Experience with Git-based version control, branching, pull requests, and CI/CD practices.

• Ability to perform unit testing, data validation, debugging, and production issue troubleshooting.

• Strong analytical, problem-solving, and root-cause analysis skills.

• Good communication and collaboration skills to work effectively with business analysts, architects, senior engineers, and cross-functional teams.

• Understanding of machine learning libraries, genie space, agent bricks is an added advantage.