Databrick Architect

Roles and Responsibilities

• Design, develop, and maintain large-scale data pipelines using Databricks.
  
• Collaborate with cross-functional teams to gather requirements and deliver high-quality solutions.
  
• Develop complex SQL queries to extract insights from massive datasets stored in Data Lakes.
  
• Implement data quality checks and validation processes to ensure accuracy and integrity of the data.
  
• Troubleshoot issues related to pipeline performance, scalability, and reliability.
  

Desired Candidate Profile

• 13-20 years of experience in designing and developing big data architectures on Azure cloud platform.
  
• Strong expertise in Databricks, Datafactory, Data Lake concepts with hands-on experience working with Spark (Scala) programming language.
  
• Proven track record of implementing end-to-end big data solutions involving ETL/ELT processes using various tools like Pyspark, Hive etc.