Data Engineer

Responsibilities:

• Design, build, and scale data pipelines that power our Healthcare Data Fabric and AI-driven applications.
• Work with data processing frameworks like Apache Spark, Databricks and dbt to transform and manage large-scale multi-modal healthcare data.
• Engineer data models and hybrid retrieval systems across Iceberg, Hive, and Delta Lake, in conjunction with vector and graph databases to support analytical and agentic queries.
• Build and optimise real-time and batch processing frameworks for high-performance data access.
• Collaborate with teams to ensure seamless data flow while ensuring and implementing data quality, observability pipelines and lineage tracking.

Requirements:

• Strong programming skills in Python (preferred) and Go.
• Hands-on experience with Spark, Databricks, and dbt for building and managing data workflows at scale.
• Familiarity with schema evolution, metadata management, and data versioning within Iceberg, Hive, or Delta Lake environments.
• Hands-on experience with SQL/NoSQL databases, vector databases, and graph stores (Neptune, Neo4j, Kuzu).
• Solid understanding of data modelling for analytical (OLAP) and transactional (OLTP) systems.