Senior Data Engineer

In this role, you will design, build, maintain, and support data pipelines and core data capabilities on Prudential Singapore's Unified Data Platform (UDP). You will deliver reliable and auditable data products supporting business operations, analytics, AI, and commissions calculations.

As part of this dynamic role, you will work closely with business, finance, technology, data, and platform teams to transform source data into trusted and reusable datasets across the organization.

Job Description:

• Design, build, test, deploy, and maintain batch and near-real-time data pipelines on the UDP platform.
• Develop data ingestion, transformation, enrichment, reconciliation, and publishing processes.
• Build and maintain data processing for commissions calculations, including rules, rates, adjustments, incentives, deductions, and calculation outputs.
• Implement reusable data models and curated datasets aligned with enterprise and UDP standards.
• Ensure data accuracy, completeness, lineage, traceability, security, and auditability across pipelines and calculation processes.
• Implement automated data quality checks, controls, alerts, and exception handling.
• Optimize pipeline performance, reliability, scalability, and cost.
• Use source control, automated testing, CI/CD, environment controls, and deployment standards.
• Monitor production pipelines, investigate failures and data discrepancies, and provide operational support.
• Contribute to the enhancement and maintenance of core UDP platform capabilities, engineering standards, and reusable components.

Who we are looking for:

• Strong hands-on experience with Python, SQL, Databricks, Spark, and data pipeline development.
• Experience designing ETL/ELT solutions, data models, transformations, and workflow orchestration.
• Experience implementing complex calculation logic, reconciliation, and financial or commissions-related data processing.
• Good understanding of data quality, observability, lineage, metadata, data contracts, and access controls.
• Experience with Delta Lake, Unity Catalog, Git, automated testing, CI/CD, and cloud data platforms.
• Understanding of APIs, file-based integration, event processing, and secure data exchange.
• Good knowledge of Agile delivery, software engineering practices, and production support.
• Strong problem analysis, troubleshooting, communication, and collaboration skills.
• Insurance or financial services experience is an advantage.

Working Experience:

• At least 5 years of experience in data engineering or software engineering.
• Hands-on experience delivering and supporting production data pipelines and enterprise data platforms.

Education:

• Bachelor in Computer Science, Data Engineering, Information Systems, Engineering or equivalent.

Language:

• Fluent written and spoken English.