Data Engineer

We are looking for an experienced Data Engineer with 6-8 years of hands-on experience in designing, building, and managing scalable data pipelines, data platforms, and analytics-ready data assets. The ideal candidate should have strong expertise in data engineering, cloud platforms, ETL/ELT pipelines, data lakehouse architecture, data quality, and production-grade data solutions.

Responsibilities:

• Design, develop, and maintain scalable data pipelines for structured, semi-structured, and unstructured data.
• Build and manage data ingestion frameworks from ERP, CRM, SCADA, IoT, Historian, APIs, databases, and external data sources.
• Develop ETL/ELT workflows for data transformation, cleansing, validation, and enrichment.
• Implement data lake, data warehouse, and lakehouse architectures to support analytics, AI/ML, and reporting use cases.
• Ensure data quality, reliability, lineage, metadata management, and governance across data platforms.
• Collaborate with Data Scientists, ML Engineers, Product Managers, Business Teams, and BI Developers to deliver data solutions.
• Optimise data pipelines for performance, scalability, cost, and reliability.
• Build reusable data assets, data marts, and curated datasets for business intelligence and AI use cases.
• Support deployment, monitoring, troubleshooting, and improvement of production data pipelines.

Requirements:

• Strong hands-on experience in SQL, Python, PySpark, and data pipeline development.
• Experience with cloud data platforms such as Azure Data Factory, Azure Synapse, Databricks, ADLS, AWS Glue, Redshift, S3 BigQuery, Cloud Functions, Data Flow, or Snowflake.
• Good understanding of data lake, data warehouse, lakehouse, data modelling, and medallion architecture.
• Experience in batch and real-time/streaming data processing.
• Knowledge of Apache Spark, Kafka, Airflow, dbt, Delta Lake, or similar technologies.
• Experience with data quality, validation, metadata, lineage, and governance frameworks.
• Exposure to CI/CD, Git, DevOps practices, and production deployment of data pipelines.
• Understanding of APIs, file formats, databases, and enterprise integration patterns.