CHENNAI · FULLTIME
Lead Data Engineer
Kumaran Systems
Chennai · onsite · Posted 67d ago
About a minute. You'll make an account on the way through.
agent · working
- Read this posting0.8s
- Found the form on Foundit · 11 questions to answer1.5s
- Decided the answers for all of them3.2s
- Tailor your résumé to this role
- Waiting for you
Section · 01
About this role
Role Overview
We are seeking a skilled Data Engineer to design, build, and optimize Azure-based data pipelines and lakehouse solutions. The ideal candidate will deliver secure, reliable datasets with strong governance, automation, and documentation while collaborating across teams in an Agile environment. This role requires strong expertise in Databricks, Python, Spark, data pipelines, and Azure ecosystem, along with a focus on performance, scalability, and data quality.
Key Responsibilities
- Design and develop scalable data pipelines using Azure and Databricks
- Build and maintain ETL/ELT workflows from diverse data sources
- Experience with Unity Catalog migration and data governance.
- Develop data transformations using Python, Spark, and Pandas
- Implement Delta Lake and Medallion architecture (Bronze/Silver/Gold layers)
- Manage data storage using Azure Data Lake Storage (ADLS)
- Design and optimize SQL queries and data models (star/dimensional schema)
- Implement CI/CD pipelines for data workflows using Azure DevOps or GitHub
- Ensure data quality, validation, monitoring, and observability
- Implement secure data access using Azure Entra ID (SSO/RBAC)
- Maintain clear documentation for pipelines and data contracts
- Participate in Agile ceremonies, code reviews, and team collaboration
- Support and mentor junior team members
Required Skills & Experience
- 8+ years of experience in Data Engineering
- Strong experience with Azure Databricks (Python, Spark, Pandas)
- Experience with Azure Data Factory (ADF) for orchestration
- Hands-on experience with Delta Lake and Medallion architecture
- Strong knowledge of Azure Data Lake Storage (ADLS)
- Experience with Unity Catalog migration and data governance
- Strong proficiency in SQL and performance optimization
- Experience in ETL/ELT and data modeling (star/dimensional schema)
- Experience with CI/CD tools (Azure DevOps or GitHub Enterprise)
- Understanding of Azure Entra ID for security and access control
- Experience in data validation, testing, and monitoring pipelines
- Familiarity with Agile/Scrum methodologies (Jira/Confluence)
- Strong communication, documentation, and collaboration skills
Preferred Experience :
- Experience with Databricks DevOps (cluster configuration, secrets management)
- Experience with Azure Functions (Python or C#)
- Exposure to Synapse SQL pools, dbt, or Delta Live Tables
- Experience in Financial Services domain
- Ability to work independently and contribute to team outcomes
- Strong focus on secure, production-grade data solutions
- Commitment to data reliability, performance, and governance
- Continuous learning mindset and adaptability
Responsible use of Generative AI tools with validation and guardrails
Sourced from foundit
Section · 02
Skills
Section · Company
About Kumaran Systems
Kumaran Systems
IT Services & Consulting
765
employees
1990
36 years old
Chennai, Tamil Nadu
US
₹10.6L PA avg
Avg at Kumaran Systems
About
Industries
Employee ratings
10 reviews
Culture
3.3
Career growth
3.4
Work-life
3.4
Employees rate it well for
Find them on