Senior Data Engineer

Role: Senior Data Engineer

Experience Level: 4+ Years

Work location: Bangalore/Mumbai/Trivandrum

Role & responsibilities

• Data and AI Engineer responsible for implementing AI data pipelines that bring together structured, semi-structured and unstructured data to support AI and Agentic solutions.
• This Includes pre-processing with extraction, chunking, embedding and grounding strategies to get the data ready.
• Real-Time Data Streaming: Build and maintain scalable and robust real-time data streaming pipelines using technologies such as Apache Kafka, AWS Kinesis, Spark streaming, or similar.
• Develop AI-driven systems to improve data capabilities
• Implement efficient Retrieval-Augmented Generation (RAG) architectures and integrate with enterprise data infrastructure
• Develop data domains and data products for various consumption archetypes including Reporting, Data Science, AI/ML, Analytics etc.
• Ensure the reliability, availability, and scalability of data pipelines and systems through effective monitoring, alerting, and incident management.
• Model domain entities, relationships, and business logic in knowledge graphs (e.g., Neo4j, Amazon Neptune, RDF).
• Implement scalable semantic layer with dynamic query translation to deliver real time insights for conversational analytics.
• Collaborate closely with DevOps and infrastructure teams to ensure seamless deployment, operation, and maintenance of data systems. Develop graph database solutions for complex data relationships supporting AI systems.

Required Experience & Skills:

• Real-Time Data Streaming Apache Kafka, AWS Kinesis, Spark Streaming
• RAG (Retrieval-Augmented Generation) Design and implement RAG architectures integrated with enterprise data infrastructure
• Data & AI Pipeline Development End-to-end pipelines for structured, semi-structured, and unstructured data (extraction, chunking, embedding, grounding)
• Knowledge Graphs & Graph Databases Neo4j, Amazon Neptune, RDF; query languages like Cypher and SPARQL
• GenAI & LLM Integration Apply GenAI solutions, integrate semantic layers with LLM platforms
• Semantic Layer Design Scalable semantic layers with dynamic query translation for real-time conversational analytics
• Cloud Platforms AWS, GCP
• Synthetic Data Generation — Simulate rare events/edge cases and integrate with automated testing frameworks
• Data Domains & Data Products — For reporting, Data Science, AI/ML, and Analytics consumption
• Reliability Engineering — Monitoring, alerting, fault tolerance, disaster recovery
• DevOps & CI/CD for Data — AI-augmented self-healing pipelines, automated data quality frameworks
• Domain Knowledge — Insurance-specific data use cases and enterprise governance standards