Sr Consultant Data Engineer

Role & responsibilities

• Design and implement data pipelines using Microsoft Fabric to extract, transform, and load (ETL) data from various sources.
• Develop and manage scalable data models in Microsoft Fabrics OneLake and integrate with Power BI datasets.
• Build interactive dashboards and reports using Power BI within Microsoft Fabric.
• Implement data governance policies, RBAC using Functions and RLS, manage access controls within Microsoft Fabric to ensure compliance and security.
• Leverage Microsoft Fabric’s Lakehouse for ingesting structured and unstructured data into data lake for analytics and AI use cases.
• Optimize data pipelines in ADF and queries, and reports to ensure high performance across analytics workloads.
• Collaborate with data engineers, analysts, and business stakeholders to define and implement data strategies using Microsoft Fabric.
• Integrate and create ETL pipelines in Microsoft Fabric with Azure services like Azure Data Factory, Azure Synapse, and Azure Machine Learning.
• Implement real-time data processing and streaming analytics using Microsoft Fabric’s built-in capabilities.
• Run pyspark ETL queries on notebooks and perform data transformation using pyspark script.