Databricks Engineer

Role & responsibilities

• Design, develop, and maintain scalable ETL/ELT pipelines using Databricks.
• Build and optimize data workflows using Apache Spark (PySpark/Scala).
• Develop and manage data lakes and lakehouse architectures using Delta Lake.
• Ingest, transform, and process structured and unstructured data from multiple sources.
• Implement data quality, governance, and security best practices.
• Optimize Spark jobs and Databricks workloads for performance and cost efficiency.
• Collaborate with data architects, analysts, and business stakeholders to deliver data solutions.
• Integrate Databricks with cloud-native services across Azure, AWS, or GCP.
• Monitor, troubleshoot, and resolve data pipeline and platform issues.
• Participate in architecture reviews and provide technical recommendations.

Preferred candidate profile

• 4+ years of Data Engineering experience.
• Hands-on experience with Databricks Platform.
• Strong expertise in PySpark/Spark SQL.
• Proficiency in Python and SQL.
• Experience with Delta Lake, Databricks Workflows, and Unity Catalog.
• Experience in ETL/ELT pipeline development.
• Knowledge of data modeling concepts.
• Experience with Git, CI/CD, and Agile methodologies.
• Understanding of data warehousing concepts.