Lead Generative AI Engineer (Databricks Genie)

Job Title

Senior / Lead Generative AI Engineer (Databricks Genie)

Role Overview

We are looking for a Senior to Lead Generative AI Engineer with 7 to 12 years of experience to drive the architecture, development, and scaling of our enterprise compound AI systems. In this role, you will design and implement production-ready agentic workflows natively within the Databricks ecosystem.

You will lead the creation of intelligent automation solutions using DB Genie, Agent bricks, and advanced multi-agent orchestration frameworks, while ensuring absolute data security through Unity Catalog.

Key Responsibilities

• Agentic Architecture: Design, build, and deploy robust multi-agent orchestration systems capable of handling complex, non-linear enterprise workflows and decision-making logic.
• Autonomous Workflows: Leverage Agent bricks to create reusable, task-specific autonomous agents that securely interact with corporate knowledge bases.
• Conversational Data Interfaces: Implement and optimize DB Genie to deliver seamless, highly accurate text-to-SQL capabilities and conversational analytics across enterprise datasets.
• Governance & Security: Architect all AI pipelines natively within Unity Catalog to enforce strict data lineage, fine-grained access control, and regulatory compliance.
• Enterprise LLMOps & Monitoring: Establish end-to-end evaluation frameworks and build custom DBX metrics views to track model safety, system latency, token costs, and hallucinatory behavior in production.
• Technical Leadership: Guide the engineering team on LLMOps best practices, conduct code reviews, and collaborate with cross-functional data teams to optimize feature stores and vector databases.

Required Technical Skills

• Databricks Ecosystem: Deep, production-tested experience with Databricks workspaces, Delta Lake, MLflow, and Unity Catalog.
• GenAI & Agentic Frameworks: Proven hands-on experience building multi-agent systems using DB Genie, Agent bricks, and orchestration layers (such as LangGraph, CrewAI, AutoGen, or native Databricks Agent frameworks).
• Observability: Strong expertise in setting up LLM tracking and operational dashboards using DBX metrics views.
• Core Engineering: Expert-level proficiency in Python and advanced SQL. Strong experience with deep learning and embedding frameworks (PyTorch, Hugging Face).
• Data & Cloud Infrastructure: Solid understanding of vector databases, retrieval-augmented generation (RAG) optimization patterns, and CI/CD pipelines for machine learning (LLMOps).

Experience & Qualifications

• Experience: 712 years of professional software/data engineering experience, with at least 23 years dedicated exclusively to building and scaling Generative AI or Agentic solutions in production.
• Education: Bachelors or Masters degree in Computer Science, Data Science, Artificial Intelligence, or a related quantitative field.
• Leadership Traits: Proven ability to mentor mid-level engineers, take complete ownership of complex technical roadmaps, and effectively translate business needs into technical architecture.