BENGALURU · FULLTIME
EY - GDS Consulting - AI and DATA - Kafka Engineer - Senior
Ernst Young
Bengaluru · onsite · Posted 5d ago
Sourced from
Undisclosed5–7 yrsfulltimeBengaluru
Your match
?
Sign in to see your match score, skill gaps & tailored resume.
Section · 01
About this role
Job Summary
EY is seeking a highly skilled Data Engineering and Data Architecture professional with strong expertise in Databricks, Kafka, AWS, and modern Data Lakehouse architecture. The ideal candidate will have hands-on experience designing and implementing scalable data platforms, building enterprise-grade data pipelines, and delivering analytics-ready data products. The candidate should possess deep technical knowledge across the data engineering lifecycle, from ingestion and transformation to modelling and consumption, while driving delivery through engineering best practices, automation, and stakeholder collaboration.
Key Responsibilities - Design and implement modern Data Lakehouse platforms using Databricks on AWS
- Build ETL/ELT pipelines
- Implement Kafka streaming solutions
- Design Fact and Dimension models
- Implement CI/CD using Azure DevOps and GitHub
- Provide technical leadership
- Design and implement modern cloud-based Data Lakehouse platforms using Databricks on AWS.
- Define and implement scalable Raw, Silver, and Gold layer architectures.
- Architect end-to-end data solutions supporting reporting, BI, advanced analytics, and AI use cases.
- Establish data platform standards, governance controls, and engineering best practices.
- Design, develop, and optimize large-scale ETL/ELT pipelines.
- Build high-performance Spark-based data processing solutions.
- Develop reusable and metadata-driven ingestion and transformation frameworks.
- Ensure data quality, reliability, scalability, and operational efficiency across data pipelines.
- Implement data validation, reconciliation, and monitoring mechanisms.
- Handling semi-structured data formats such as JSON, XML.
- Design and implement real-time data ingestion frameworks using Kafka.
- Build scalable streaming solutions integrating Kafka with Databricks and AWS services.
- Optimize streaming workloads for performance and resilience.
- Design conceptual, logical, and physical data models.
- Create dimensional models, Fact tables, and Dimension tables for enterprise reporting and analytics.
- Implement data models aligned with business and analytical requirements.
- Implement CI/CD pipelines for data engineering workloads.
- Leverage Azure DevOps, GitHub, Databricks Asset Bundles (DAB), and automation frameworks.
- Promote version control, code reviews, testing, and release management practices.
- Collaborate with business stakeholders, architects, and delivery teams to translate requirements into scalable solutions.
- Provide technical leadership and mentoring to engineering teams.
- Contribute to solution design workshops, estimations, and architecture reviews.
- Databricks (Expert Level or Databricks Champion)
- Apache Spark (PySpark / Spark SQL)
- Delta Lake
- Kafka Streaming
- Semi-structured data processing such as JSON, XML
- AWS Cloud Services
- ETL/ELT Design and Development
- Data Lakehouse Architecture
- Data Modeling and Dimensional Modeling
- Data Warehousing Concepts
- SQL and Performance Optimization
- Real-Time Data Processing
- Data Pipeline Orchestration
- GitHub
- Azure DevOps
- CI/CD Implementation
- Databricks Asset Bundles (DAB)
- Agile Delivery Methodology
- Strong problem-solving and analytical skills.
- Excellent communication and stakeholder management capabilities.
- Ability to work in a global delivery environment.
- Strong ownership mindset with focus on quality and client value.
- Demonstrated leadership and mentoring capabilities.
- Experience designing enterprise-scale Data Lakehouse solutions.
Sourced from naukri
Let the agent run this one for you.
Tailored resume, auto-apply, and referral lookup — in under 2 minutes.
Section · 02
Skills
AutomationAzure DevopsAgileApache SparkCI/CD Pipelinesanalytical thinkingApacheAws Servicesamazon web servicesAgile Deliveryarchitectural patternsAnalyticsBankingCapital MarketscloudArchitectureAdvanced AnalyticsMicrosoft AzureArtificial Intelligence
Section · Company
About Ernst & Young
Ernst Young
Management Consulting
44k+
employees
2002
24 years old
Gurgaon / Gurugram, Haryana
India
₹15.3L PA avg
Avg at Ernst & Young
About
At EY, our purpose is Building a better working world. The insights and quality services we provide help build trust and confidence in the capital markets and in economies the world over.
Industries
Management Consulting
Employee ratings
10 reviews
3.4/ 5
Culture
1.9
Career growth
2.3
Work-life
2.5
Employees rate it well for
Job Security36Skill Development35
Find them on