Developer III - DevOps Engineering

Job Title: Platform Engineer - AWS & Databricks Overview

We are looking for an experienced Platform Engineer to design, automate, and manage scalable cloud infrastructure and data platforms on AWS and Databricks. This role requires strong expertise in cloud engineering, Infrastructure as Code (IaC), platform governance, automation, and DevOps practices to deliver secure, reliable, and cost-effective solutions.

The ideal candidate will collaborate with data engineering, analytics, security, and business teams to build and maintain modern cloud platform capabilities that support enterprise data and analytics workloads.

Key Responsibilities Cloud Platform Engineering

• Design, deploy, and manage cloud infrastructure and platform services on AWS and Databricks.
• Build scalable, secure, and highly available environments that support data and analytics workloads.
• Ensure platform solutions align with security, governance, and operational best practices.

Infrastructure as Code (IaC)

• Develop and maintain infrastructure using Infrastructure as Code tools such as Terraform, AWS CloudFormation, or AWS CDK.
• Automate infrastructure provisioning, deployment, and configuration management processes.
• Promote infrastructure standardisation and repeatable deployment patterns.

AWS Platform Management

• Manage and support core AWS services including:
  • Amazon S3
  • IAM
  • Lambda
  • EC2
  • KMS
  • VPC
  • API Gateway
  • Route 53
  • Secrets Manager
• Implement networking, security, and access management best practices across cloud environments.

Databricks Platform Administration

• Provision and manage Databricks workspaces, compute resources, SQL Warehouses, and governance frameworks.
• Support data platform operations, performance optimisation, and access management.
• Implement governance controls to ensure secure and compliant platform usage.

DevOps & Automation

• Build and maintain CI/CD pipelines using modern Git-based workflows.
• Automate deployment, configuration, and operational processes to improve efficiency and reliability.
• Support platform releases through robust DevOps practices.

Security, Governance & Compliance

• Implement cloud security best practices, including access controls, encryption, secrets management, and network security.
• Support governance, compliance, auditing, and risk management initiatives.
• Ensure cloud platforms meet organisational security and regulatory requirements.

Platform Reliability & Monitoring

• Monitor platform health, performance, and availability.
• Troubleshoot infrastructure and platform-related issues.
• Support observability, performance tuning, incident management, and operational excellence initiatives.

Collaboration & Stakeholder Engagement

• Work closely with engineering, analytics, architecture, and business teams to deliver platform capabilities.
• Participate in technical discussions, solution reviews, and platform improvement initiatives.
• Communicate technical solutions and recommendations effectively to stakeholders.

Required Qualifications Experience

• 5+ years of experience in Platform Engineering, Cloud Engineering, DevOps, Infrastructure Engineering, or related roles.
• Experience managing cloud-native platforms in production environments.

Technical Skills

• Strong expertise in AWS cloud services and platform engineering.
• Hands-on experience with Databricks platform administration and governance.
• Experience with Infrastructure as Code (Terraform, AWS CloudFormation, AWS CDK, or equivalent).
• Strong Python programming and automation skills.
• Experience implementing and supporting CI/CD pipelines.

Preferred Qualifications

• Experience with Databricks Workflows and platform automation frameworks.
• Knowledge of Apache Spark, Delta Lake, and modern data platform architectures.
• Experience with observability, monitoring, and cloud cost optimisation practices.
• Exposure to AI/ML platform enablement and API integrations.
• Experience working in multi-account AWS environments.
• Understanding of cloud financial governance and cost management frameworks.

Education

• Bachelors or Masters degree in Computer Science, Information Technology, Engineering, or a related discipline.

Key Skills

• Databricks
• AWS Cloud Platform
• Terraform
• AWS CloudFormation
• Infrastructure as Code (IaC)
• Python
• CI/CD
• DevOps
• Cloud Security
• Platform Engineering