Lead / Senior Data Engineer

We are growing our Data & AI practice, which delivers cloud data platforms, data modernisation and migration, analytics and AI-enabled engineering for global enterprise clients. You will work in cross-functional teams on long-term engagements, using modern cloud technologies and AI-assisted engineering practices.

Role Summary

Builds, tests and runs data pipelines and data platforms end to end on AWS from ingestion to business-ready data – including migration of legacy data workloads. One JD covering three levels.

Roles & Responsibilities

• Build ingestion, transformation and data-quality pipelines (batch and incremental)
• Model and manage data in layered lakehouse / warehouse architectures
• Develop with dbt, PySpark and SQL; orchestrate with Apache Airflow
• Migrate and refactor legacy SQL, workflows and scripts to modern tools, using AI-assisted conversion
• Configure data platform components on client environments; deploy through CI/CD
• Tune performance and cost; support parallel runs and go-lives
• Senior / Lead: set coding standards, review code, plan work, mentor engineers

Mandatory Skills

• Python, PySpark, advanced SQL
• dbt and Apache Airflow
• AWS: S3, Glue, Athena, EMR; Redshift a plus
• Data modelling and upsert / deduplication patterns
• Git and CI/CD practices
• Data-quality testing

Nice to Have

• Apache Iceberg / Delta / Hudi
• Customer data platforms (CDP) or marketing data
• Legacy workflow / ETL tools and SQL dialects (Presto, Hive)
• Streaming (Kafka, Kinesis)
• AI coding assistants (Claude Code, GitHub Copilot, Amazon Q Developer)

Also Expected

Clear communication with remote client stakeholders across time zones; Comfortable using AI coding and productivity tools, and reviewing AI-generated work critically; Agile ways of working; ownership and accountability. Certifications: AWS Certified Data Engineer – Associate (preferred).