Databricks Data Engineer

We are seeking a Databricks Data Engineer to join our growing data engineering team in Pune, India. CES is open to considering qualified candidates for either a direct-hire, full-time opportunity or a contract-to-hire arrangement, depending on the candidate's experience, availability, and preferred employment structure.

This role will play a key part in a large-scale modernization initiative to migrate a complex, enterprise-grade Microsoft SQL Server data warehouse ecosystem to the Databricks Lakehouse Platform.

The ideal candidate has strong hands-on experience across Databricks data engineering capabilities, with a core focus on building scalable, reliable data pipelines and analytics workloads. Exposure to Databricks AI/ML capabilities is a plus.

Key Responsibilities

• Design, build, and optimize scalable data pipelines using Databricks, Apache Spark, Delta Lake, and Unity Catalog.
• Participate in the migration of an approximately 20TB on-premises Microsoft SQL Server data warehouse to Databricks.
• Convert and modernize hundreds of SQL Server tables, thousands of SSIS jobs, and downstream SSRS and SSAS workloads.
• Re-engineer legacy SSIS ETL processes into Databricks notebooks, workflows, and orchestration frameworks.
• Support the migration and redesign of SSAS cube-based analytics into Databricks SQL, Delta tables, and modern semantic models.
• Implement data quality, validation, reconciliation, audit, and migration controls throughout the modernization process.
• Optimize Spark performance, partitioning strategies, query performance, and cost efficiency across large-scale datasets.
• Apply data governance, security, lineage, and access-control standards using Unity Catalog.
• Collaborate with analytics, BI, and AI/ML teams to enable downstream reporting, analytics, and advanced data use cases.
• Contribute to reusable frameworks, technical documentation, engineering standards, and platform best practices.

Qualifications

• Bachelor's degree in Computer Science, Engineering, or a related field.
• 4–7 years of overall data engineering experience.
• Hands-on experience with Databricks on AWS preferred; Azure Databricks experience is also acceptable.
• Strong proficiency in Spark, including PySpark and/or Scala, and SQL.
• Proven experience participating in the migration of on-premises SQL Server data warehouses to cloud-based data platforms.
• Experience converting SSIS-based ETL pipelines into Spark-based data engineering solutions.
• Experience working within structured migration or enterprise modernization initiatives rather than solely lift-and-shift projects.
• Strong understanding of data warehousing concepts, dimensional modeling, and analytical workloads.
• Experience with Delta Lake, incremental processing patterns, and data versioning.
• Familiarity with Databricks Workflows, Jobs, and production-grade deployments.
• Practical experience with performance tuning and large-volume data processing.

Preferred Skills

• Experience modernizing SSAS cube-based reporting solutions into Lakehouse-native semantic models.
• Exposure to Databricks SQL Warehouses and BI integrations, with Power BI experience preferred.
• Working knowledge of Medallion architecture and cloud-native data engineering practices aligned with Databricks best practices.
• Familiarity with MLflow, feature engineering, feature stores, or other AI/ML capabilities within Databricks.
• Databricks certification is a plus.

Position Details

• Location: Pune, Maharashtra
• Employment Type: Direct Hire or Contract-to-Hire
• Work Arrangement: Hybrid, with a minimum of three days per week in the Pune office, preferably Tuesday, Wednesday, and Thursday

Additional Information

Customized Energy Solutions India offers competitive compensation along with performance-based incentives tied to individual and company performance. Our benefits are designed to support both professional and personal well-being and include comprehensive medical insurance coverage for associates and eligible dependents, generous paid time off, and opportunities for professional development through training and skill-building programs.

We are committed to creating an environment where associates are supported, challenged, and provided opportunities to grow and develop their careers over the long term.