Data Engineer

Responsibilities:

• Data Pipeline Operationalisation: Lead the design and maintenance of scalable ETL/ELT pipelines to ingest and transform data from diverse sources into central data warehouses.
• Automation and CI/CD: Implement and manage robust CI/CD pipelines specifically for data assets; automate extraction, testing, and deployment processes to increase velocity and reduce manual intervention.
• Infrastructure-as-Code (IaC): Utilise tools such as Terraform to manage and version cloud infrastructure, ensuring environment parity across development, staging, and production.
• Data Observability and Reliability: Proactively monitor pipeline health and data quality (freshness, volume, and distribution). Design and build automated alerting systems to resolve bottlenecks.
• Security and Compliance: Implement and enforce data security measures, including IAM roles, encryption, and access controls, ensuring all pipelines meet GDPR and CCPA compliance.
• Platform Support and Collaboration: Partner with Data Scientists and Analysts to facilitate data availability and provide infrastructure support for complex modelling and insights.
• Process Optimisation: Identify and execute improvements in data engineering workflows, adopting software engineering best practices for stateful data flows.

Requirements:

• Execution and SLA Excellence: Consistently meets and exceeds Service Level Agreements (SLAs) by streamlining data workflows and maintaining high system availability. Translates business requirements into measurable performance targets.
• Operational Governance: Implements rigorous operational standards and best practices, leveraging the ITIL framework, specifically Incident, Problem, and Change Management, to minimise downtime and ensure predictable system behaviour.
• Global Collaboration and Follow-the-Sun Support: Experienced in working with distributed "around-the-sun" teams, ensuring seamless handovers, consistent documentation, and 24/7 operational continuity across different time zones.
• Innovation and Adaptability: Adapts DevOps and DataOps strategies to handle the unique challenges of data statefulness and rapidly evolving cloud-native technologies.
• Strategic Problem Solving: Independently diagnoses and resolves complex architectural bottlenecks; moves beyond "quick fixes" to identify root causes and prevent recurring systemic issues.
• Communication and Stakeholder Management: Clearly articulates system health, operational risks, and technical requirements to both technical peers and executive leadership, bridging the gap between data science needs and engineering standards.

Requirements:

• Experience: 6 years of professional experience in DataOps, Data Engineering, or DevOps, with a significant focus on large-scale data systems.
• Technical Proficiency: Expert-level Python and SQL skills.
• Orchestration and Transformation: Hands-on experience with modern orchestration tools (e. g., Apache Airflow, Mage, or Prefect) and transformation layers (e. g., dbt).
• Cloud and Containers: Deep experience with AWS (or GCP/Azure) and containerization technologies such as Docker and Kubernetes.
• DevOps Tooling: Proficiency with Git for version control and Terraform for infrastructure management.
• Data Architecture: Advanced knowledge of modern data warehouse/lakehouse environments (e. g., Snowflake, Databricks).

Preferred Qualifications:

• Experience in the gaming industry or high-volume consumer data environments.
• Advanced certifications in AWS or relevant Data Engineering specialisations.
• Experience with real-time streaming technologies (e. g., Kafka, Kinesis).