Lead Data Engineer

Job Summary

We are seeking a talented and experienced Data AI Engineer with strong Azure cloud competencies to join our dynamic team.

Responsibilities

• Design and implement streaming data pipelines integrating Kafka with Databricks using Structured Streaming.
• Architect and maintain Medallion Architecture with well-defined Bronze, Silver, and Gold layers.
• Implement efficient ingestion using Databricks Autoloader for high-throughput data loads.
• Work with large volumes of structured and unstructured data, ensuring high availability and performance.
• Apply performance tuning techniques such as partitioning, caching, and cluster resource optimization.
• Collaborate with cross-functional teams (data scientists, analysts, business users) to build robust data solutions.
• Establish best practices for code versioning, deployment automation, and data governance.

Qualifications

• B.Tech/BCA or any Graduation

Skills

• Streaming data pipelines
• Medallion Architecture
• Databricks Autoloader

Work Mode

Work from Office

Disclaimer: This job posting has been aggregated from external source. Role details, content, and availability are subject to change. Applicants are advised to confirm the latest information directly on the company website before applying.