Data Engineer

Job Summary

The Data Engineer is responsible for the design, development, and implementation of scalable data solutions across the Azure Databricks platform supporting AUNZ. This role focuses on building and maintaining data pipelines using Databricks (Unity Catalog, DABs, DLT), PySpark, and SQL, delivering enterprise-scale data transformation, and supporting Power BI reporting through well-modelled silver/gold layer datasets. The Data Engineer works closely with the Head of Data & Analytics to execute technical scoping and delivery against business requirements.

Responsibilities

• KRA 1 Data Pipeline Development - Design, build, test, and maintain data pipelines (ELT) using Databricks, Delta Live Tables, and Databricks Asset Bundles, according to business and technical requirements.
• KRA 1 Data Pipeline Development - Manage data workflows and orchestration; ensure accuracy and availability of all ELT processes.
• KRA 1 Data Pipeline Development - Design and deliver silver/gold layer data models within the medallion architecture to support downstream reporting.
• KRA 2 Platform & Governance - Implement secure platforms with Unity Catalog - access controls, data lineage, and discovery.
• KRA 2 Platform & Governance - Apply CI/CD and DevOps practices (GitHub Actions) for deployment, testing, and release management.
• KRA 2 Platform & Governance - Play a key role in automation, building solutions against architectural best practice.
• KRA 3 Solution Delivery & Documentation - Propose technical designs and develop integrations, including application data connections via APIs.
• KRA 3 Solution Delivery & Documentation - Execute data migrations as scoped and directed by the Head of Data & Analytics.
• KRA 3 Solution Delivery & Documentation - Maintain clear documentation of pipelines, data models, and technical logic within the engineering layer.
• KRA 3 Solution Delivery & Documentation - Provide technical input during requirements scoping sessions with the Head of Data & Analytics.

What youll need

• 3+ years experience designing and building scalable distributed data pipelines and dimensional data models.
• 3+ years experience in Python and SQL.
• Experience using the Databricks platform and PySpark is required (Unity Catalog, DABs, DLT).
• Experience with Azure data services - ADLS Gen2, Azure Key Vault, Azure SQL.
• Demonstrable understanding of CI/CD and Agile practices, including unit/integration testing.
• Exposure to AI & ML technologies is a plus.

Who you are

• Youre open: we are inclusive and collaborative; we encourage the free exchange of ideas; we respect and celebrate diverse views. We are open-minded: to new ideas, new partnerships, new ways of working.
• Youre optimistic: we approach all that we do with confidence: to try the new and to seek the unexpected.
• Youre extraordinary: We are stronger together: through collaboration we achieve the amazing. We are creative leaders and pioneers of our industry; we provide extraordinary every day.

Disclaimer: This job posting has been aggregated from external source. Role details, content, and availability are subject to change. Applicants are advised to confirm the latest information directly on the company website before applying.