Data Engineer

Roles and Responsibilities

• Design, develop, and maintain scalable, enterprise-grade AI agents and high-volume ELT/ETL pipelines using Python, PySpark, Databricks, Kafka, and FastAPI.
• Implement and deploy GenAI agents using Google ADK and Google Flash 2.5+ LLMs to power application automation and workflow optimization via Human-in-the-Loop (HIL) design.
• Build and maintain data federation layers for Lambda and Data Mesh architectures using Starburst to enable machine learning, deep learning, and NLP use cases.
• Develop, deploy, and automate resilient microservice integrations supporting data-intensive applications on Kubernetes and OpenShift cloud-native platforms.
• Integrate agentic AI tools such as Devin.AI and GitHub Copilot via Model Context Protocol (MCP) and advanced prompt engineering to maximize development velocity.
• Enforce data quality, security, and risk compliance across the data lifecycle, ensuring adherence to regulatory standards and internal control policies.
• Establish CI/CD pipelines, unit testing frameworks, and engineering best practices to support high-availability platform deployments.

Preferred Candidate Profile

Experience:

• Overall Experience: 8+ years in large-scale software engineering or data platform development.
• Relevant Experience: 5+ years of hands-on technical lead experience in Python, PySpark, and Databricks pipeline architectures.
• Domain Experience: Background in Banking, Financial Services, Retail Products (Cards, Mortgages, Deposits), or Wealth Management risk domains is strongly preferred.

Technical Expertise:

• Core Data Engineering: Python, PySpark, Databricks, SQL, Kafka, Data Modeling, Data Warehousing.
• AI & Microservices: GenAI Agents, Google ADK, LLM Integrations (Flash 2.5+), MCP, FastAPI, Microservices, Prompt Engineering.
• Platform & Infrastructure: Data Mesh, Starburst, OpenShift, Kubernetes, Docker, CI/CD.