Azure Data Engineer

Role & responsibilities

• Core technical skill sets - Azure Databricks, SQL, Python, PySpark, Delta Lake, Azure Data Factory and Storage Services
• Good to have CI/CD experience for data platforms, MLFlow
• Independently design, develop, optimize, support end-to-end data engineering solutions involving both real-time and batch ingestion pipelines
• Work with Analysts and business on requests to build/enhance downstream data pipelines to support analytics, reconciliation systems and Power BI Reports
• Good understanding of medallion architectures so that he can support teams to optimize the platform for costs and performance
• Engineer reliable integrations across systems such as trading platforms, Salesforce Marketing Cloud, CRM, Finance & Operational databases
• Implementation of Unity Catalog to enable our systems for AI on Databricks