Senior Data Engineer – Databricks

Job Summary We are looking for a

Senior Data Engineer with strong expertise in

Databricks, SQL, Python, Spark, dbt, and Airflow to design, develop, and optimize scalable data pipelines. The ideal candidate should have experience in modern cloud data platforms, data modeling, ETL/ELT development, and preparing analytical datasets for BI reporting. The role involves building robust batch and near real-time data pipelines while ensuring data quality, governance, security, and performance. Key Responsibilities

• Design, develop, and maintain scalable data pipelines using Databricks, Spark, Python, SQL, dbt, and Airflow.
• Build batch and near real-time ETL/ELT pipelines.
• Develop reusable data transformation frameworks.
• Design dimensional data models (Star Schema, Fact & Dimension Tables, SCD).
• Integrate data from APIs, SaaS applications, event streams, flat files, logs, and operational databases.
• Work with Databricks Workflows or equivalent orchestration tools.
• Optimize SQL queries, Spark jobs, and data pipelines.
• Prepare datasets for BI tools such as Power BI, Tableau, Looker, or Superset.
• Implement data quality, metadata management, lineage, and monitoring.
• Manage access control, row-level security, and sensitive data governance.
• Collaborate with Data Analysts, BI Developers, and Business Stakeholders. Required Skills
• Databricks
• Apache Spark
• PySpark
• SQL
• Python
• dbt
• Apache Airflow
• Databricks Workflows
• ETL/ELT
• Data Engineering
• Data Pipelines
• Data Modeling
• Star Schema
• Fact Tables
• Dimension Tables
• Slowly Changing Dimensions (SCD)
• Data Warehousing Preferred Skills
• Snowflake
• Azure Synapse
• Amazon Redshift
• Google BigQuery
• PostgreSQL
• Delta Lake
• Unity Catalog
• Azure Data Factory (ADF)
• Kafka
• Event Streaming
• Power BI
• Tableau
• Looker
• Superset
• Git
• CI/CD