Senior Data Engineer

As part of the Big Data B2B program, OBS set up a shared Big Data platform and a Data Lake for Use cases exploration and industrialization.

We are looking for Senior Data Engineer 5-8 years of experience in building Data Pipelines with below KRA(S):

• Automate, Industrialize the build and development tasks.
• Lead discussion sessions with stakeholders.
• Participate in all areas of the data Engineering life-cycle,requirements gathering and data mapping, systems design, data ingestion development, preparing data mapping documentation, testing and deployment, post implementation support and monitoring.
• Resolve and troubleshoot problems and complex issues.
• Provide innovative solutions to complex business problems.
• Report to and work closely with project teams and Business Analysis team on project delivery status.
• Prepare progress update and status report.
• Provide operational support, ongoing maintenance and enhancement after implementation as part of Run Management Activities.
• Implementing Best Data Integration Practices.

• Good understanding of Big Data Ecosystem with frameworks like HADOOP, SPARK.
• Good Understanding of Data Layers : Bronze, Silver, Gold.
• Good experience handling large volume data as well as both structured/unstructured data in Streaming and Batch modes : Kappa, Medallion Architecture, Lambda Architecture
• High Coding proficiency in at least one modern programming language: Python, Java or Scala.
• Hands on Experience on NiFi , Hive, SQL/HQL, Oozie, Pyspark, Data Bricks
• Good Understanding of Data Integration Patterns : Medallion and Kappa
• Good understanding of KAFKA, Rabbit MQ.
• Good understanding of API concepts : REST and also microservices architecture
• Good Understanding of Devops toolings and Concepts like Kubernetes, Jenkins, Gitlab
• Good to have understanding of ELK stack .
• Good to have understanding of Monitoring tools like Prometheus etc.
• Good Understanding of Cloud architecture is must.
• Good to be professionally certified in any of the Hyperscalers preferably Azure.
• Azure Data Engineering Stack:
• Data Ingestion: Uses Azure Data Factory (ADF) for scheduled batch ETL/ELT orchestration, alongside
  • Azure Event Hubs and IoT Hub for high-throughput, real-time streaming data (such as app logs,
  • clickstreams, or IoT telemetry).
• Storage Layer: Uses Azure Data Lake Storage (ADLS) Gen2 and Azure Blob Storage as a scalable,
  • cost-effective foundation to store raw, curated, and enriched data (often structured into a Bronze,
  • Silver, and Gold medallion architecture).
• Data Processing Transformation: Uses Azure Databricks (powered by Apache Spark) for heavy big-data engineering, machine learning, and stream-batch processing.
• Data Warehousing Analytics: Uses Azure Synapse Analytics as a unified platform to query, model, and serve transformed data to Business Intelligence (BI) tools like Power BI.
• Good knowledge on Azure Data Fabric.
• Good Experience of AGILE methods (Scrum, Kanban)
• Good understanding of JIRA.
• Good to have understanding of Power BI Reporting.
• Understanding of Data Modelling is must
• Understanding of Open Digital Architecture and TMF principles is preferably.
• Understanding of Tools Like DSS, Jupiter Notebook is preferably.

You bring a can-do attitude, tackle challenges head-on and challenge the status quo with new and innovative ideas.

Regardless of your age, gender identity, race, ethnic origin, religion/belief, sexual orientation, marital status, neurotype, disability, veteran status or appearance, we encourage diversity within our teams because it is a strength for the collective and a vector of innovation. Orange Group is a disabled-friendly company and equal opportunity employer: dont hesitate to tell us about your specific needs.