Data Engineer - MLOPS

Role & responsibilities :

• Design, develop, and maintain scalable batch and streaming pipelines that ingest data from vehicles, sensors, APIs, files, databases, and cloud services.
• Build reusable ETL/ELT frameworks for transforming raw data into curated, analytics-ready and ML-ready datasets.
• Design data models and storage patterns across data lakes, lakehouses, warehouses, and SQL and NoSQL systems.
• Develop APIs and data services that enable secure, efficient access to raw and processed data.
• Implement automated data validation, reconciliation, schema checks, lineage, metadata, and quality monitoring across the data lifecycle.
• Improve pipeline reliability through observability, alerting, retry and recovery patterns, testing, and systematic root-cause analysis.
• Optimise distributed processing, storage, and query performance while balancing scalability, availability, and cloud cost.
• Apply security, privacy, access-control, retention, and governance requirements to data products and platforms.
• Build and maintain automated CI/CD pipelines, infrastructure-as-code, containerised workloads, and deployment practices for data services across AWS and/or Azure.
• Partner with AI/ML and MLOps teams to deliver high-quality datasets, features, and interfaces for model development, validation, and production workflows.
• Collaborate with global product, platform, embedded, validation, security, and domain teams to translate requirements into maintainable data solutions.
• Contribute to architecture reviews, coding standards, technical documentation, peer reviews, and the continuous improvement of engineering practices.

Preferred candidate profile :

• Bachelors or masters degree in Computer Science, Information Technology, Data Science, Engineering, or a related discipline, or equivalent practical experience.
• 510 years of professional experience in data engineering, software engineering, or a closely related field.
• Strong programming skills in Python and advanced working knowledge of SQL.
• Hands-on experience designing and operating production ETL/ELT pipelines for large-scale datasets; experience at petabyte scale is preferred.
• Experience with data modelling, relational databases, NoSQL systems, and cloud object storage.
• Experience with distributed data processing technologies such as Apache Spark or an equivalent managed platform.
• Practical experience with at least one public cloud platform (AWS or Azure) and its data, compute, storage, security, and monitoring services.
• Experience with workflow orchestration and scheduling tools such as Apache Airflow, Azure Data Factory, AWS Step Functions, or equivalent.
• Proficiency with Git, automated testing, code review, and CI/CD practices.
• Good understanding of data quality, schema evolution, idempotency, partitioning, performance tuning, observability, and failure recovery.
• Ability to troubleshoot complex data and platform issues using a structured, evidence-driven approach.
• Strong communication and collaboration skills, with experience working across functions and geographies.

Good to Have:

• Experience with streaming and messaging platforms such as Apache Kafka, Kinesis, or Event Hubs.
• Experience with Docker, Kubernetes, serverless architectures, and infrastructure-as-code tools such as Terraform or CloudFormation.
• Exposure to data lakehouse, transformation, or table-format technologies such as Databricks, dbt, Delta Lake, Apache Iceberg, or equivalent.
• Experience with monitoring and visualisation tools such as Grafana, Kibana, OpenSearch/ELK, Qlik Sense, or Power BI.
• Familiarity with MLOps platforms, feature or dataset management, and data workflows for model training and validation.
• Experience with ADAS, autonomous driving, automotive, IoT, telemetry, sensor, image, video, or other high-volume engineering datasets.
• Working knowledge of Java, Scala, C, or C++ where integration with platform or embedded systems is required.
• Experience mentoring engineers or leading the technical delivery of a workstream.