Databricks + Pyspark

AIA-Pune

Job Summary

We are seeking an experienced Architect with 8 to 12 years of experience to join our team in a hybrid work model. The ideal candidate will have expertise in Databricks Unity Catalog Admin Delta Live Pipelines Databricks SQL Databricks Delta Lake Databricks Workflows and PySpark. This role involves designing and implementing data solutions that align with our companys objectives ensuring efficient data management and analysis.

Responsibilities

• Design and implement scalable data architecture solutions using Databricks technologies to meet business requirements.
• Oversee the integration of Databricks Unity Catalog Admin to ensure seamless data governance and security.
• Provide expertise in Delta Live Pipelines to automate and streamline data processing workflows.
• Develop and optimize Databricks SQL queries for efficient data retrieval and analysis.
• Implement Databricks Delta Lake to ensure data reliability and consistency across the organization.
• Manage Databricks Workflows to automate complex data processing tasks and improve operational efficiency.
• Utilize PySpark to develop and maintain data processing applications that support business analytics.
• Collaborate with cross-functional teams to understand data requirements and deliver solutions that drive business value.
• Ensure data solutions are aligned with industry best practices and company standards.
• Monitor and troubleshoot data systems to ensure optimal performance and reliability.
• Provide technical guidance and support to team members on Databricks-related projects.
• Contribute to the continuous improvement of data architecture processes and methodologies.
• Stay updated with the latest advancements in Databricks technologies to enhance data solutions.
  
  

Qualifications

• Possess strong experience in Databricks Unity Catalog Admin and Delta Live Pipelines.
• Demonstrate proficiency in Databricks SQL and Databricks Delta Lake.
• Have hands-on experience with Databricks Workflows and PySpark.
• Exhibit excellent problem-solving skills and attention to detail.
• Show ability to work collaboratively in a hybrid work environment.
• Display strong communication skills to effectively convey technical concepts.
• Hold a bachelors degree in Computer Science Information Technology or a related field.