Databricks data engineer (Azure/AWS)- PAN INDIA

Greetings from Tata Consultancy Services (TCS)

What we are looking for: Databricks data engineer (Azure/AWS)

Location : PAN INDIA

Interview Mode : Virtual mode (Microsoft Teams)

Experience : 5-10 Years

• Strong Experience and understanding of Data warehouse, Data Lake platforms architecture and design aspects
• Design and develop scalable end-to-end ETL/ELT pipelines on Azure/AWS Databricks using PySpark, Spark SQL and Delta Lake.
• Implement medallion (BronzeSilverGold) Lakehouse architectures with automated ingestion, schema evolution and performance optimisation.
• Build streaming and batch ingestion using Auto Loader, Lakeflow/Structured Streaming, Delta Live Tables (DLT) and Azure Data Factory.
• Enforce data governance, lineage, access control and cataloguing through Unity Catalog, RBAC and Delta Sharing.
• Optimise Spark workloads for cost and performance (partitioning, caching, cluster sizing, query tuning) across large datasets.
• Support legacy-to-Lakehouse migrations (Hadoop/DataStage/RDBMS) leveraging accelerators such as Lakebridge and reusable frameworks.
• Establish CI/CD for Databricks assets using Azure DevOps / Git, and build automated data validation and testing utilities.
• Collaborate with architects, AI/ML specialists and client teams on solutioning, demos, PoCs and production rollouts.

TECHNICAL SKILLS