Data Bricks Administrator

Role & responsibilities

We are seeking a Senior Data Platform Engineer to support the administration, automation, governance, security, and optimization of our enterprise Databricks Lakehouse platform on AWS.

This is a hands-on technical role responsible for maintaining and enhancing the Databricks platform, implementing infrastructure automation, enforcing compute and security standards, supporting cost-governance initiatives, and enabling reliable Data Engineering and Data Science workloads.

The ideal candidate will have strong hands-on experience with Databricks platform administration, AWS cloud services, Terraform-based infrastructure automation, and DevOps practices. The individual will evaluate emerging Databricks capabilities through proof-of-concept implementations and contribute to reusable platform patterns and standards.

This role will collaborate closely with Data Engineering, Data Science, Cloud Platform, Security, and Governance teams to ensure the Databricks environment remains secure, reliable, scalable, and cost-efficient.

Skills Required

• Hands-on experience administering Databricks workspaces, including cluster policies, job policies, compute governance, networking, security, and platform integrations
• Experience implementing Unity Catalog governance and access management
• Experience with Delta Lake architecture, performance optimization, and troubleshooting
• Strong hands-on experience with AWS IAM, VPC networking, S3, AWS KMS, CloudWatch, PrivateLink, cross-account access, and cloud security
• Experience using Terraform for Databricks and AWS infrastructure
• Experience with infrastructure-as-code design patterns and platform automation frameworks
• Experience with Databricks CLI, Databricks REST APIs, Python, and Bash scripting
• Experience developing CI/CD pipelines and automated deployment and release processes
• Experience with Git-based version control and development workflows
• Experience with monitoring, logging, observability, incident management, and root cause analysis
• Experience with Apache Spark, PySpark, SQL optimization, streaming frameworks, data pipeline orchestration, data modeling, and ETL best practices
• Databricks Certified Data Engineer or Databricks Platform Architect certification
• AWS Certified Solutions Architect certification
• Experience designing enterprise-scale Lakehouse platforms
• Experience supporting large-scale machine learning workloads
• Experience implementing data governance frameworks

Technical Skills

Databricks Platform

• Databricks workspace administration
• Cluster and job policy design
• Unity Catalog governance
• Delta Lake architecture
• Databricks REST APIs and CLI automation
• Performance optimization and troubleshooting

Preferred candidate profile

8-10 Yrs of experience

AWS Cloud Platform

Strong hands-on experience with:

• AWS IAM
• VPC networking and security
• S3 data lake architecture
• AWS KMS encryption
• CloudWatch monitoring
• PrivateLink connectivity
• Cross-account access architecture
• Enterprise cloud security practices

Infrastructure & Automation

• Terraform for Databricks and AWS infrastructure
• Infrastructure as Code design patterns
• Databricks CLI and API automation
• Python / Bash scripting
• Platform automation frameworks

DevOps & DataOps

• CI/CD pipeline development
• Git-based version control workflows
• Automated deployment and release processes
• Monitoring, logging, and observability frameworks
• Incident management and root cause analysis

Data Engineering & Analytics Skills

• Apache Spark / PySpark
• SQL optimization
• Delta Lake performance tuning
• Streaming frameworks
• Data pipeline orchestration
• Data modeling and ETL best practices

Preferred Qualifications

• Databricks Certified Data Engineer / Databricks Platform Architect
• AWS Certified Solutions Architect
• Experience designing enterprise-scale Lakehouse platforms
• Experience supporting large-scale ML workloads
• Experience implementing data governance frameworks