Azure databricks + Pyspark

AIA-Pune

Job Summary We are seeking a highly skilled Sr. Developer with 7 to 10 years of experience to join our team. The ideal candidate will have expertise in Databricks Unity Catalog Azure DevOps Python Databricks SQL Databricks Workflows and PySpark. Experience in Institutional Asset Management and Asset & Wealth Management is a plus. This hybrid role offers the opportunity to work on innovative projects that drive our companys success.

Responsibilities

• Develop and implement robust data solutions using Databricks Unity Catalog to enhance data management capabilities.
• Collaborate with cross-functional teams to integrate Azure DevOps into the development lifecycle ensuring seamless deployment processes.
• Utilize Python to create efficient and scalable data processing scripts that support business objectives.
• Design and optimize Databricks SQL queries to extract valuable insights from large datasets contributing to data-driven decision-making.
• Manage and automate data workflows using Databricks Workflows improving operational efficiency and reducing manual intervention.
• Employ PySpark to process and analyze big data delivering actionable insights that align with organizational goals.
• Work closely with stakeholders to understand business requirements and translate them into technical solutions that drive value.
• Ensure data quality and integrity by implementing best practices and conducting thorough testing of developed solutions.
• Provide technical guidance and support to junior developers fostering a collaborative and knowledge-sharing environment.
• Stay updated with the latest industry trends and technologies to continuously improve development practices and outcomes.
• Contribute to the companys purpose by developing solutions that enhance asset and wealth management processes impacting society positively.
• Participate in code reviews and provide constructive feedback to maintain high coding standards and improve team performance.
• Document all development processes and solutions to ensure transparency and facilitate future enhancements.

Qualifications

• Possess strong experience in Databricks Unity Catalog Azure DevOps Python Databricks SQL Databricks Workflows and PySpark.
• Demonstrate a solid understanding of Institutional Asset Management and Asset & Wealth Management domains.
• Exhibit excellent problem-solving skills and the ability to work independently in a hybrid work model.
• Show proficiency in developing scalable and efficient data solutions that meet business needs.
• Have a proven track record of delivering high-quality projects within deadlines and budget constraints.

Certifications Required

• Databricks Certified Data Engineer Associate Microsoft Certified: Azure Developer Associate