Data Engineer

Role Synopsis

The Data Engineer will be responsible for developing and maintaining our data infrastructure, as well as developing and implementing data pipelines to move data from our various sources into our data warehouse, leveraging Microsoft Fabric and related technologies. The Data Engineer will work closely with the other Data Engineers, Power BI Developers to help deliver data projects and BAU tasks.

Key Accountability

• Develop and maintain our data infrastructure, including data warehouses, data lakes, OneLake, and data pipelines.
• Design, build, and maintain data pipelines and workflows using Microsoft Fabric, Synapse Data Engineering, and Azure Data Factory (ADF).
• Work with other Data Engineers and stakeholders to understand their data needs and develop solutions to meet those needs.
• Develop and implement data quality assurance processes to ensure the accuracy and reliability of our data.
• Automate data processing tasks to improve efficiency, including CI/CD for Fabric deployments.
• Work with other Data Engineers to implement new data products and services.

Skills Required

• 5+ years of experience in data engineering.
• Experience with data modelling and data warehousing.
• Experience with data quality assurance.
• Experience with scripting languages, such as Python and SQL.
• Strong problem-solving and analytical skills.
• Experience of MSSQL, Fabric, Power BI, and Azure Data Factory (ADF) highly desirable.

Key Technical Skills

• Core Platform: Microsoft Fabric, OneLake, Synapse Data Engineering, Synapse Data Science
• Languages: Python (PySpark, pandas, NumPy, Scikit-Learn), SQL, Delta Lake
• Workflows: Fabric Notebooks, Data Factory Pipelines, CI/CD for Fabric