Junior Data Engineer

The Data Engineer designs, builds, and maintains data pipelines that power analytics engagements. Working alongside Data Scientists and Analysts, this role ensures reliable, performant, and well-governed data infrastructure is available for client delivery across APAC markets. The ideal candidate has hands-on experience with real-time and batch data pipelines in cloud environments, ideally within financial services.

Key Responsibilities

• Design and implement batch and real-time data pipelines using Python/PySpark and cloud-native tools
• Build and maintain ETL/ELT workflows to move and transform large volumes of payments and transaction data
• Integrate with client data sources including APIs, flat files, databases, and streaming platforms
• Ensure data quality, lineage, and governance standards are met across all pipeline outputs
• Optimise query performance and manage schema evolution in data warehouses
• Collaborate with Data Scientists to prepare feature-engineered datasets for model training and scoring
• Document pipeline architecture, data flows, and operational runbooks Support troubleshooting, incident response, and SLA adherence for production pipelines

Requirements

• 3–6 years of hands-on data engineering experience — mandatory
• Proficiency in Python and/or PySpark for data pipeline development — mandatory
• Experience with at least one cloud data platform: AWS (S3, Glue, Redshift), GCP (BigQuery, Dataflow), or Azure (ADF, Synapse) — mandatory
• Strong SQL skills for data transformation and validation — mandatory
• Experience with workflow orchestration tools: Apache Airflow or equivalent — mandatory
• Familiarity with streaming platforms (Kafka or Kinesis) preferred
• Understanding of data warehousing concepts, star/snowflake schemas, and partitioning strategies Bachelor’s degree in Computer Science, Engineering, or equivalent

Tech Stack Python, PySpark, Apache Airflow, Apache Kafka, AWS (S3 / Glue / Redshift) or GCP (BigQuery / Dataflow) or Azure (ADF / Synapse), SQL, dbt, Github

Nice to Have Experience with payments or financial transaction data pipelines; Databricks certification; exposure to data mesh or data lakehouse architectures; Snowflake or Delta Lake experience