Module Lead - Data Bricks

5+ Years

JD for Data Engineer

Responsibilities

• Migrate existing code and workloads from Databricks to Azure Synapse .
• Refactor pandas-based data transformation logic into scalable PySpark / Azure Synapse Spark implementations
• Develop, enhance, and support data products that serve business and analytics needs.
• Write and optimize SQL for data extraction, transformation, validation, and reconciliation.
• Design and build reliable data pipelines and data processing workflows.
• Work with structured and semi-structured data to support downstream data products.
• Validate migrated code and data outputs to ensure accuracy and consistency.
• Troubleshoot issues across data pipelines, scripts, and Azure-based environments.
• Collaborate with data analysts, engineers, and business stakeholders to understand requirements and deliver solutions.
• Document technical solutions, migration changes, and data product logic.
• Support code reviews, testing, and deployment activities.

Required Technical Skills

• Strong hands-on experience with Azure Synapse
• Strong hands-on experience with Spark/PySpark
• Hands-on experience with Microsoft Fabric
• Proficiency in SQL and Python
• Experience working with pandas and re-factoring into PySpark and Azure Synapse Spark implementations
• Strong experience using Git and Azure DevOps for source control, pull requests, CI/CD pipelines, and deployment automation
• Strong experience with contributing to reusable engineering patterns and shared libraries
• Solid understanding of Azure fundamentals, including storage, compute, security, and identity
• Sold understanding of data modeling and data transformation concepts
• Solid experience in performance optimization

Preferred Technologies, Platforms, and Tools

• Microsoft Fabric
• SAP ECC / S/4HANA
• Oracle ERP
• Salesforce
• Parquet, JSON, CSV data formats
• GitHub, Azure DevOps, Terraform

Preferred / Nice-to-Have

• Experience with Databricks notebooks and migration of workloads to Synapse
• Familiarity with Azure Data Factory or Synapse Pipelines
• Exposure to Microsoft Fabric and modern cloud data platforms
• Knowledge of data quality checks, performance tuning, and monitoring.
• Experience building data products in a cloud environment.
• Familiarity with version control, collaborative development, and DevOps practices, and tooling (Git, Azure DevOps)
• Experience with JSON, CSV, Parquet, or other common data formats
• Experience with observability practices, including monitoring data pipelines, performance tuning, and investigating failures