Sr Data Engineer

Job Summary

An exciting opportunity to work at the Trading & Supply- Shell Energy Software Engineering & Analytics organization. We are looking for a talented Senior Data Engineer with great interpersonal skills to join our team.

Senior Data Engineer is an individual with proven experience in adhering to Data Engineering best practices, delivering Data related optimal & performance driven solutions which helps business making quick decision while generating value. They are expected to have continuous improvement mindset, driving value-based process, product/application optimization.

Responsibilities

• Design, develop, and maintain scalable, high-performance data engineering solutions that support business decision-making and generate measurable value.
• Build and optimize end-to-end data pipelines using Azure Data Factory (ADF), Azure Data Lake Storage (ADLS), Databricks, PySpark, Python, and SQL.
• Develop data engineering solutions for the Energy rading & Supply business, leveraging domain knowledge of ETRM ( Endur, and Nucleus).
• Implement data modeling strategies to support enterprise reporting, analytics, and operational requirements.
• Drive continuous improvement initiatives by optimizing data products, processes, applications, and platform performance.
• Apply performance tuning and optimization techniques using PySpark and SQL to improve data processing efficiency and scalability.
• Design and support enterprise data platforms, Master Data Management (MDM), Data Catalog, and analytics solutions.
• Collaborate with cross-functional teams in an Agile environment, ensuring adherence to data engineering best practices, quality standards, and governance requirements.
• Manage source control, CI/CD, and release management processes using GitHub, Azure DevOps, and related tools.
• Work with relational and non-relational databases to support data integration, transformation, and analytics initiatives.
• Partner with business and technical stakeholders to translate Energy Trading business requirements into scalable data solutions.
• Contribute to innovation initiatives, including the evaluation and adoption of emerging technologies and AI-driven solutions.

Requirements

• Bachelors degree in Engineering, Computer Science, Information Technology, or a related field; MCA qualification is also acceptable.
• With at least 8 - 10 years of experience in Data Engineering, with a proven track record of delivering enterprise-scale data solutions.
• Strong experience in the Trading & Supply domain, including Endur and Nucleus.
• Deep understanding of the Energy Trading business domain and experience building data solutions that support trading operations and decision-making.
• Expertise in Microsoft Azure technologies, including:
  • Azure Data Factory (ADF)
  • Azure Data Lake Storage (ADLS)
  • Azure Databricks
  • Data Pipeline Development
  • Azure Fabric
• Strong hands-on experience with PySpark, Python, Advanced SQL and Data Modeling
• Experience working with both relational and non-relational database technologies.
• Proven expertise in SQL query optimization and PySpark performance tuning.
• Experience with source control and DevOps practices, including GitHub, Azure DevOps, build pipelines, and release management.
• Hands-on experience implementing enterprise data platforms, Master Data Management (MDM), Data Catalog, and analytics solutions.
• Experience working in Agile delivery environments and applying engineering best practices.
• Databricks Certified Associate certification required.
• Excellent verbal and written communication skills.

Preferred Qualifications

• Experience working with data products and platforms within the Energy sector.
• Knowledge of ETRM (Energy Trading and Risk Management) systems, particularly Endur, and its integration with enterprise energy applications.
• Experience implementing AI-powered solutions and platforms, such as AI Foundry and Databricks Genie.
• Hands-on experience with Microsoft Fabric and/or Power BI.
• Exposure to advanced analytics, data governance, and modern cloud-based data architectures.

Disclaimer: This job posting has been aggregated from external source. Role details, content, and availability are subject to change. Applicants are advised to confirm the latest information directly on the company website before applying.