Azure Databricks Engineer

Roles and Responsibilities :

• Design, develop, test, deploy and maintain large-scale data pipelines using Azure Databricks.
• Collaborate with cross-functional teams to gather requirements and deliver high-quality solutions on time.
• Develop complex SQL queries to optimize database performance and troubleshoot issues in Delta Lake databases.
• Implement machine learning models using PySpark and large language models for business intelligence applications.

Job Requirements :

• 5-10 years of experience in IT Services & Consulting industry as an Engineer or similar role.
• Strong expertise in Azure Databricks, including designing, developing, testing, deploying and maintaining large-scale data pipelines.
• Proficiency in working with Delta Lake databases and implementing efficient query optimization strategies.
• Experience with implementing machine learning models using PySpark and large language models.