Sr DATA ENGINEER - Azure Databricks

Job Summary

We are seeking a hands on Data Engineer to build reliable scalable Databricks pipelines. You will translate requirements into production ELT or ETL jobs Delta Lake tables and reusable components. You will refine reference patterns optimize Spark for performance and cost apply Unity Catalog best practices orchestrate workflows and deliver high quality well documented code that empowers teams.

Responsibilities

• build reliable scalable Databricks pipelines
• translate requirements into production ELT or ETL jobs Delta Lake tables and reusable components
• refine reference patterns
• optimize Spark for performance and cost
• apply Unity Catalog best practices
• orchestrate workflows
• deliver high quality well documented code that empowers teams

Requirements

• Core qualifications include Bachelors degree in CS Engineering or related field and 5 years experience in software or data engineering with 2 years working with Databricks and Spark.
• Strong skills in Python SQL and PySpark required along with AWS or Azure knowledge.
• Experience with Databricks Lakehouse Workflows SQL and dbt preferred.
• Solid understanding of Lakehouse and data warehousing architecture AI or ML pipeline support and familiarity with Terraform or CloudFormation valued.
• Experience ingesting data using APIs preferred or strong understanding of APIs.
• Exposure to dbt and MLOps or DevOps is a plus.

Skills

• Databricks
• Spark
• Python
• SQL
• PySpark
• AWS
• Azure
• dbt
• APIs

Disclaimer: This job posting has been aggregated from external source. Role details, content, and availability are subject to change. Applicants are advised to confirm the latest information directly on the company website before applying.