SENIOR DATA ENGINEER - Azure Databricks

Location: SMILES 3-Bengaluru-Bengaluru

Years of Experience: 5-8 Years

Job Summary: We are seeking a skilled Databricks Platform Administrator with a strong background in Azure Databricks to manage and optimize our data analytics platform. The ideal candidate will have extensive experience in administering Databricks environments, ensuring high availability, performance, and security. You will collaborate with data engineers, data scientists, and other stakeholders to support data-driven decision-making across the organization.

Responsibilities:

• Administer and manage Azure Databricks environments, ensuring optimal performance and reliability.
• Monitor and troubleshoot Databricks clusters, jobs, and notebooks to ensure smooth operations.
• Implement security best practices, including user access management and data governance.
• Collaborate with data engineering and data science teams to support their analytics needs.
• Develop and maintain documentation for Databricks configurations, processes, and best practices.
• Stay updated with the latest Databricks features and Azure services to enhance platform capabilities.
• Provide training and support to team members on Databricks usage and best practices.
• Participate in capacity planning and performance tuning of the Databricks environment.

Mandatory Skills:

• Proven experience with Azure Databricks administration.
• Strong understanding of data engineering concepts and practices.
• Experience with Spark, SQL, and data processing frameworks.
• Familiarity with Azure cloud services and architecture.
• Knowledge of security and compliance standards in cloud environments.

Preferred Skills:

• Experience with data visualization tools such as Power BI or Tableau.
• Familiarity with CI/CD practices and tools for data pipelines.
• Knowledge of Python, Scala, or R for data processing and analysis.
• Experience with monitoring and logging tools for cloud environments.

Qualifications:

• Bachelor's degree in Computer Science, Information Technology, or a related field.
• Relevant certifications in Azure or Databricks are a plus.
• Strong analytical and problem-solving skills.
• Excellent communication and collaboration abilities.

If you are passionate about data and have the required skills, we invite you to apply for this exciting opportunity to join our team!