Lead Data Engineer
Randstad Digital · Remote · hybrid · Posted 20d ago
About a minute. You'll make an account on the way through.
About this role
Lead the design, development, and optimization of scalable enterprise-grade data platforms on Microsoft Azure using PySpark, Azure Databricks, Delta Lake, and streaming/time-series data technologies.
Key Responsibilities
Design and implement Lakehouse architectures, lead enterprise adoption of Medallion Architecture and Data Mesh principles, develop optimized ETL/ELT pipelines, implement real-time streaming solutions, optimize Spark workloads and Azure Databricks cluster utilization, enforce data governance and quality, and lead a team while collaborating with stakeholders.
Requirements
PySpark, Azure Databricks, Delta Lake, Medallion Architecture, Spark SQL, Kafka, Azure Event Hubs, Spark Structured Streaming, Delta Live Tables, ADLS Gen2, Azure Data Factory, Azure Event Hub, Azure Functions, Azure Synapse Analytics, Azure Key Vault, Azure Monitor, Lakehouse architectures, Data Mesh, Data Platform Architecture, ETL/ELT data pipelines, Event-driven Architectures, Batch and streaming workload optimization, Partitioning Strategies, Shuffle Operations, Caching, Broadcast Joins, Adaptive Query Execution, Delta File Optimization, Spark workload optimization, Team leadership, Architecture reviews, Code reviews, Mentoring, Stakeholder collaboration
Sourced from naukri
Skills
About Randstad Digital

Randstad Digital
Recruitment
Diemen
NL
₹16.2L PA avg
Avg at Randstad Digital
₹14L – ₹20L PA
This role
About
Industries
Employee ratings
10 reviews
Culture
2.1
Career growth
1.7
Work-life
2.3
Employees rate it well for
Find them on