Databrick Engineer
Workforce Solution · Mumbai · onsite · Posted 27d ago
Undisclosed4–6 yrsfulltimeSourced from
About a minute. You'll make an account on the way through.
About this role
Key Responsibilities
- Design, develop, and maintain scalable data pipelines using Azure Databricks and PySpark.
- Build and optimize ETL/ELT processes for ingesting data from multiple source systems.
- Develop data transformation logic and implement business rules using Spark frameworks.
- Work closely with business stakeholders, data analysts, and architects to understand data requirements.
- Implement and manage Delta Lake architecture for reliable and high-performance data processing.
- Optimize Databricks jobs, notebooks, clusters, and Spark code for performance and cost efficiency.
- Develop and support data ingestion frameworks using Azure Data Factory (ADF) and other integration tools.
- Monitor production workloads, troubleshoot failures, and ensure data availability.
- Implement data quality checks, validations, and reconciliation processes.
- Ensure adherence to data governance, security, and compliance standards.
- Participate in code reviews, technical design discussions, and best practice implementation.
- Support production deployments and provide L2/L3 support for critical data pipelines.
Sourced from naukri
Skills
Databricks EngineerDatabricksPysparkAzure Data FactoryAzure Databricks