Senior Data & AI Engineer

Position Summary

We are looking for a passionate Senior Data AI Engineer to design, build, and scale modern data and AI-powered solutions that enable advanced analytics, reporting, and intelligent business applications. The ideal candidate combines deep expertise in cloud data engineering, Databricks, and AWS with hands-on experience in Generative AI, Large Language Models (LLMs), Retrieval-Augmented Generation (RAG), Vector Databases, and AI Agents architectures. This role will be responsible for delivering trusted, scalable, and analytics-ready data products while driving innovation through modern engineering practices, AI-assisted development, and cloud-native technologies.

Key Responsibilities

• Design, develop, and support scalable batch, streaming, and real-time data pipelines.
• Build and maintain analytics-ready data products, data lakehouse solutions, and curated datasets.
• Develop ETL/ELT frameworks using Databricks, PySpark, SQL, and AWS data services.
• Create and optimize data models, data marts, and enterprise data products.
• Ensure data quality through validation, monitoring, reconciliation, and automated testing.
• Implement data governance, lineage, security, access controls, and compliance standards.
• Design and develop APIs, reusable data services, and engineering accelerators.
• Leverage LLMs and AI-assisted development tools to improve engineering productivity and operational efficiency.
• Build and support RAG pipelines, vector embedding frameworks, semantic search solutions, and AI-powered applications.
• Develop and deploy AI Agents using frameworks such as LangChain, LangGraph, CrewAI, AutoGen, or similar technologies.
• Implement CI/CD pipelines, GitHub workflows, Infrastructure as Code, and automated deployment processes.
• Collaborate with Data Architects, Data Scientists, Analysts, and Product Owners to deliver business-driven solutions.
• Mentor team members and drive engineering standards, best practices, and innovation initiatives.

Skills Competencies

• Expert-level proficiency in Python, PySpark, SQL, and distributed data processing frameworks.
• Strong expertise in Databricks, including Delta Lake, Unity Catalog, Workflows, and Lakehouse architecture.
• Deep knowledge of the AWS Data Analytics ecosystem, including Glue, Lambda, S3, Athena, Redshift, Lake Formation, CloudFormation, and related services.
• Hands-on experience with Generative AI, Large Language Models (LLMs), Retrieval-Augmented Generation (RAG), Vector Databases, and AI Agent development.
• Experience leveraging AI-assisted engineering tools such as Claude Code, GitHub Copilot, Microsoft Copilot, or similar platforms to accelerate development and operational efficiency.
• Experience with Agentic AI orchestration frameworks such as LangChain, LangGraph, CrewAI, AutoGen, or equivalent technologies.
• Strong expertise in Git, GitHub Actions, CI/CD pipelines, DevOps practices, Infrastructure as Code, and AWS DevOps automation.
• Experience designing and developing APIs, microservices, and cloud-native data solutions.
• Solid understanding of data governance, metadata management, data lineage, security, privacy, and enterprise data platforms.
• Experience implementing data quality frameworks, monitoring, observability, and operational excellence practices.
• Exposure to BI and visualization platforms such as Tableau, Power BI, or equivalent tools for downstream analytics consumption.
• Strong analytical, problem-solving, communication, and stakeholder management capabilities.
• Proven ability to collaborate effectively with both technical and business stakeholders in a cross-functional, agile environment.
• Self-driven with an ownership mindset and the ability to thrive in a fast-paced, product-oriented organization.

Qualifications Experience

• Bachelors or Masters degree in Computer Science, Engineering, Data Science, Information Systems, or a related field.
• 5-8 years of hands-on experience in Data Engineering, Software Engineering, or Cloud Data Platforms.
• SME-level expertise in Databricks and modern Lakehouse architectures.
• Proven experience building enterprise-scale data pipelines and data products on AWS and/or Azure.
• Strong experience with real-time and batch data ingestion frameworks.
• Experience developing APIs and cloud-native data services.
• Hands-on experience in one or more of the following areas: AI/ML, Generative AI, Agentic AI, Retrieval-Augmented Generation (RAG), Vector Search, and LLM-powered applications.
• Experience working in Agile, product-oriented environments with end-to-end ownership.
• Ability to lead complex technical initiatives and mentor engineering teams.
• Prior experience in Life Sciences, Healthcare, or other regulated industries is a plus.
• Certifications in AWS, Databricks, Azure, Google Cloud, or related cloud and data technologies are a strong plus.

Good to Have

• Experience with Vector Databases such as Pinecone or Databricks Vector Search.
• Knowledge of MLOps, LLMOps, AI Governance, and Responsible AI practices.
• Experience building enterprise-scale AI Agents and RAG-based solutions.
• Familiarity with modern AI orchestration and automation frameworks.
• Experience working with global teams in a product-centric delivery model.

If you come across a role that intrigues you but doesn t perfectly line up with your resume, we encourage you to apply anyway. You could be one step away from work that will transform your life and career.