Databricks Developer

Job Summary

We are seeking an experienced Senior Databricks Developer with 5+ years of experience in designing and implementing scalable data engineering solutions. The ideal candidate will have strong expertise in big data processing, data pipeline development, and cloud-based data platforms, with hands-on experience using Databricks and distributed data processing frameworks like Apache Spark.

The role involves building robust data pipelines, optimizing large-scale data processing workloads, and collaborating with data engineers, analysts, and business stakeholders to enable data-driven decision-making.

Job Type: Full time

Key Responsibilities

• Design, develop, and maintain scalable data pipelines using Databricks.
• Build and optimize large-scale data processing solutions using Apache Spark.
• Develop and manage ETL/ELT workflows for ingesting, transforming, and processing structured and unstructured data.
• Write efficient data processing scripts using Python, Scala, and SQL.
• Implement and manage data lake and data warehouse architectures.
• Optimize performance of data pipelines and Spark jobs.
• Integrate Databricks with cloud data services and storage platforms.
• Collaborate with data scientists, analysts, and engineering teams to deliver reliable data solutions.
• Implement best practices for data governance, data quality, and security.
• Troubleshoot data pipeline issues and ensure system reliability and scalability.
• Document data architecture, workflows, and operational processes.

Required Skills and Qualifications

• 5+ years of experience in data engineering or big data development.
• Strong hands-on experience with Databricks.
• Extensive experience with distributed data processing using Apache Spark.
• Proficiency in Python, Scala, and SQL.
• Strong experience building scalable ETL/ELT data pipelines.
• Experience working with large datasets in data lake or data warehouse environments.
• Experience with cloud platforms such as Microsoft Azure, Amazon Web Services, or Google Cloud Platform.
• Strong knowledge of data modeling, data architecture, and data integration concepts.
• Excellent analytical, troubleshooting, and performance optimization skills.

Disclaimer: This job posting has been aggregated from external source. Role details, content, and availability are subject to change. Applicants are advised to confirm the latest information directly on the company website before applying.