Senior Data Engineer

Job Summary

Job Title: Senior Data Engineer. Position type: Full Time. Work Location: Bangalore. Working style: Hybrid. People Manager role: No. Required education and certifications critical for the role: Any Graduate or Post-Graduate (full time). Required years of experience: Minimum 7+ years of relevant experience.

Role Overview

The Senior Data Engineer will be a key contributor to the Altair modernization program, working on the Azure-based data architecture layer built on Databricks. You will design and develop scalable data pipelines that support reinsurance analytics, Property & Casualty loss ratio calculations, and end-to-end workflow automation. As an individual contributor, you will work independently on complex data engineering challenges, collaborate with cross-functional teams including Solution Architects, Product Owners, and Business Analysts, and help establish best practices for the team. This role requires strong technical skills, the ability to understand business requirements in the reinsurance domain, and a passion for building reliable, scalable data solutions.

Responsibilities

• Data Pipeline Development & Integration: Design, develop, and maintain scalable data ingestion, transformation, and integration pipelines on Databricks using PySpark. Build robust ETL/ELT workflows to process structured and unstructured data from multiple upstream systems (Meridian, email, Workflow Studio, AB Connect placements). Implement data imports and transformations using agreed ingestion patterns and Unity Catalog. Develop data flows aligned with RAPID technology stack standards and design patterns. Support the transition of Altair capabilities to modern Azure-based architecture.
• Data Modeling & Quality: Develop and maintain data models and schemas that support reinsurance business processes and analytics. Implement data quality frameworks, validation processes, and data governance standards. Ensure data lineage, security, and compliance across all data assets. Design transformations that support Property & Casualty loss ratio calculations and statistical analysis.
• Business Requirements & Domain Understanding: Collaborate with business stakeholders, SMEs, and Business Analysts to understand reinsurance business processes. Translate business requirements into technical data solutions with clear understanding of domain context. Analyze and resolve moderately complex data challenges related to reinsurance workflows. Partner with Product Owners and Project Managers to deliver trusted data assets.
• Technical Collaboration & Best Practices: Work closely with Lead Data Engineers, Solution Architects, and QA Engineers to deliver high-quality solutions. Participate in code reviews and promote data engineering best practices. Contribute to the development of reusable components and design patterns. Support knowledge transfer activities to build in-house capabilities and reduce vendor dependency. Create technical documentation and operational procedures.
• AI & Automation: Partner with AI specialists to incorporate Azure OpenAI and LLM capabilities into data solutions. Support development of intelligent data intake capabilities that trigger workflows automatically. Contribute to AI-assisted development approaches and automation initiatives.
• Platform Optimization & Support: Monitor and optimize pipeline performance, reliability, scalability, and cost efficiency on Azure. Troubleshoot production issues, perform root-cause analysis, and implement sustainable solutions. Implement CI/CD processes and automation for data engineering workflows. Identify opportunities for continuous improvement across the data ecosystem.
• About the Project: You will be joining the RAPID team working on the Altair modernization program. The initiative involves building a modern, Azure-based Workflow Studio that integrates multiple applications and enables end-to-end process custody from data intake through client deliverables in the reinsurance domain. You will work with a collaborative team including Solution Architects, Product Owners, Project Managers, QA Engineers, and reinsurance SMEs. The program is decomposing existing Altair capabilities and re-implementing them across Databricks (data layer), Workflow Studio (process layer), and Blueprint/Remetrica (analytics layer). Other existing projects in the ecosystem include Workflow Studio, Blueprint, Remetrica, and Altair manual processes.

Qualifications

• Bachelors degree in Engineering, Computer Science, Data Engineering, Information Systems, or related field
• 7+ years of experience in data engineering, software engineering, or big data development

Technical Skills

• Databricks
• PySpark
• Python
• SQL
• Azure Data Factory
• Azure Data Lake Storage
• Unity Catalog
• Azure OpenAI
• LLM integration

Business & Analytical Skills

• Strong ability to understand and translate business requirements into technical solutions
• Solid analytical and problem-solving skills
• Understanding of statistical concepts and their application in data analysis
• Proven ability to work independently with minimal supervision
• Excellent communication and stakeholder collaboration skills
• Ability to handle multiple priorities in a fast-paced environment

Preferred Qualifications

• Experience in insurance, reinsurance, or financial services domains
• Understanding of Property & Casualty insurance, loss ratio calculations, or reinsurance analytics
• Knowledge of business process management (BPM) concepts and workflow automation
• Prior involvement in modernization or cloud migration programs
• Experience working with external vendors and distributed teams
• Azure certifications (Azure Data Engineer Associate or similar)
• Familiarity with workflow orchestration tools (Airflow, Azure Data Factory, etc.)

Disclaimer: This job posting has been aggregated from external source. Role details, content, and availability are subject to change. Applicants are advised to confirm the latest information directly on the company website before applying.