Azure Databricks Platform Admin

Key Responsibilities:

• Azure Databricks Platform Administration
• Administer and manage Azure Databricks workspaces clusters and jobs
• Configure and maintain Databricks environments Dev QA Prod
• Monitor cluster performance job execution and system health
• Manage notebooks libraries and dependencies
• Perform upgrades patches and platform maintenance
• Cluster Management Optimization
• Design and manage cluster configurations auto scaling instance types pools
• Optimize job performance resource utilization and execution times
• Troubleshoot cluster failures memory issues and job bottlenecks
• Implement best practices for Spark performance tuning
• Security Governance
• Implement and enforce data security and access controls RBAC ACLs IAM
• Manage workspace security secrets Key Vault integration and credential passthrough
• Ensure compliance with enterprise security and governance policies
• Implement data governance auditing and lineage tracking
• Azure Cloud Integration
• Integrate Databricks with Azure services such as
• Azure Data Lake Storage ADLS Gen2
• Azure Key Vault
• Azure Data Factory ADF
• Azure Synapse Analytics
• Manage connectivity and networking VNet Private Endpoints NSGs

Technical Requirements:

• Core Skills
• 10 15 years of IT experience with strong Azure Databricks administration
• Hands on expertise in Apache Spark and Databricks platform management
• Strong experience with Azure cloud services ADLS ADF Key Vault Networking
• Proven experience in production support and platform operations
• Technical Skills
• Deep knowledge of cluster configuration job scheduling and performance tuning
• Strong understanding of Spark architecture and optimization techniques
• Experience with Terraform ARM Bicep IaC tools
• Knowledge of security models IAM RBAC and data governance
• Familiarity with CI CD tools and DevOps practices
• Preferred Skills
• Experience with Delta Lake and Lakehouse architecture
• Exposure to Python PySpark Scala
• Familiarity with monitoring tools Azure Monitor Splunk
• Knowledge of multi cloud environments AWS GCP
• Experience in data engineering or ETL pipelines

Preferred Skills:

Technology->Cloud Platform->Azure Analytics Services->Azure Databricks