Data Engineer

Data Engineer

2-5 Years of Experience ( Chennai / Bangalore) ** About the role:** We are looking for a skilled Data Engineer to design, develop, and optimize scalable data solutions. The role involves building modern data pipelines, working with cloud-based data platforms, and enabling reliable data processing through automation, governance, and performance optimization. Key Responsibilities. Develop and maintain scalable data pipelines using Databricks, PySpark, and Spark SQL Build and manage Medallion Architecture (Bronze, Silver, Gold) pipelines Design batch and streaming data solutions using Delta Lake and Structured Streaming Work with DLT, Databricks Workflows, and Unity Catalog for pipeline management and governance Develop data ingestion frameworks using files, APIs, Kafka/Event Hubs, and other sources Optimize data processing for performance, scalability, and cost efficiency Support CI/CD, automation, and reusable component development Ensure data quality, validation, and compliance with governance standards Collaborate with teams to deliver reliable and business-ready data solutions Good to have: Exposure to Lakebase, ZeroBus, Lakebridge, Delta Sharing, and cloud platforms (Azure/AWS/GCP) Required Skills to Have. Strong hands-on experience in Databricks, PySpark, and Spark SQL Build and maintain Medallion Architecture (Bronze, Silver, Gold) pipelines Develop batch & streaming pipelines using Delta Lake and Structured Streaming Hands-on with DLT, Workflows, and Unity Catalog (basic governance) Experience in data ingestion (files, APIs, Kafka/Event Hubs) Focus on performance tuning, optimization, and cost efficiency Support CI/CD, automation, and reusable components development Ensure data quality, validation, and governance compliance Exposure to Lakebase, ZeroBus, Lakebridge, and Delta Sharing (good to have) Basic understanding of cloud platforms (Azure/AWS/GCP. We are looking for a skilled Data Engineer to design, develop, and optimize scalable data solutions. The role involves building modern data pipelines, working with cloud-based data platforms, and enabling reliable data processing through automation, governance, and performance optimization.

Key Responsibilities: Develop and maintain scalable data pipelines using Databricks, PySpark, and Spark SQL Build and manage Medallion Architecture (Bronze, Silver, Gold) pipelines Design batch and streaming data solutions using Delta Lake and Structured Streaming Work with DLT, Databricks Workflows, and Unity Catalog for pipeline management and governance Develop data ingestion frameworks using files, APIs, Kafka/Event Hubs, and other sources Optimize data processing for performance, scalability, and cost efficiency Support CI/CD, automation, and reusable component development Ensure data quality, validation, and compliance with governance standards Collaborate with teams to deliver reliable and business-ready data solutions Good to have: Exposure to Lakebase, ZeroBus, Lakebridge, Delta Sharing, and cloud platforms (Azure/AWS/GCP)

Required Skills to Have: Strong hands-on experience in Databricks, PySpark, and Spark SQL Build and maintain Medallion Architecture (Bronze, Silver, Gold) pipelines Develop batch & streaming pipelines using Delta Lake and Structured Streaming Hands-on with DLT, Workflows, and Unity Catalog (basic governance) Experience in data ingestion (files, APIs, Kafka/Event Hubs) Focus on performance tuning, optimization, and cost efficiency Support CI/CD, automation, and reusable components development Ensure data quality, validation, and governance compliance Exposure to Lakebase, ZeroBus, Lakebridge, and Delta Sharing (good to have) Basic understanding of cloud platforms (Azure/AWS/GCP)

Note : Kindly note that this role is open for both Chennai and Bangalore locations