Data Engineer

Job description

Key Responsibilities:

Data Modelling & Engineering:

• Design, implement, and maintain data models using Azure Data Modeling techniques to meet business needs.
• Create and optimize ETL pipelines, leveraging Azure Data Factory and Azure Databricks to process large datasets and integrate data from various sources.

Data Integration:

• Work on integrating data from multiple platforms and databases into a cohesive environment using Azure technologies, ensuring data consistency and quality.

Data Analysis & Reporting:

• Implement data engineering solutions to support reporting, analytics, and business intelligence tasks.
• Utilize Python and SQL to manipulate and analyze datasets to deliver insights and reports.

Cloud Data Services:

• Work with cloud data services within the Windows Azure ecosystem, ensuring efficient data storage, processing, and retrieval.

Collaboration:

• Collaborate with cross-functional teams, including Data Scientists, Analysts, and other Engineers, to support data-driven decision-making processes.

Performance Optimization:

• Ensure high performance and scalability of data models, pipelines, and processes to handle growing data loads efficiently.

Documentation:

• Maintain clear and up-to-date documentation for data engineering processes, models, and pipelines.

Skills and Qualifications:

Hands-on Experience:

• Minimum 5-7 years of experience in Data Engineering or related roles, with proven expertise in Azure Data Modelling, Azure Data Factory, Python, and SQL.

Technical Skills:

• Strong knowledge and hands-on experience with Azure Data Factory, Azure Databricks, and other Azure-based data services.
• Proficiency in SQL for data manipulation, extraction, and reporting.
• Experience with Python for scripting and automating data-related tasks and workflows.
• Knowledge of Windows Azure cloud infrastructure, storage, and computing services.

Data Modelling:

• Experience with Data Modelling techniques, including normalization, schema design, and data integration.

Communication & Collaboration:

• Excellent communication skills with the ability to work collaboratively in a remote team and document solutions clearly.

Problem-Solving:

• Strong analytical and troubleshooting skills to resolve data-related issues and optimize solutions.