Databricks AI Architect
Impetus Technologies · Indore · onsite · Posted 27d ago
Undisclosed10–15 yrsfulltimeSourced from
About a minute. You'll make an account on the way through.
About this role
The ideal candidate is a hands-on AI architect with deep Databricks and GenAI expertise, strong data engineering foundations, and a proven record of taking AI solutions to production at enterprise scale. Key requirements:
- 12-14 years of experience in data and AI/ML solutions, including designing and architecting large-scale (TB/PB) data platforms and production AI/ML systems.
- Deep, hands-on architecture expertise with Databricks - Spark/PySpark, Delta Lake, lakehouse/medallion architecture, Unity Catalog, Databricks Workflows, and Delta Live Tables - with strong performance and cost optimization.
- Proven experience architecting Generative AI / LLM solutions - RAG pipelines, chunking and embedding strategies, vector store design, prompt engineering, evaluation, and guardrails/responsible-AI patterns.
- Strong hands-on experience with native Databricks AI capabilities - Mosaic AI (Model Serving, Vector Search, Agent Framework), Foundation Model APIs, MLflow, Feature Store, and model governance via Unity Catalog.
- Solid ML/MLOps foundations - model lifecycle management, experiment tracking, deployment, monitoring, and drift/quality evaluation in production.
- Expert-level PySpark and Spark SQL, with strong Python for building reusable data and feature pipelines that power AI workloads (batch and streaming).
- Strong hands-on experience with AWS and Azure Cloud, including networking, IAM/security, cost optimization, and AI/ML services; comfortable architecting across cloud data and AI services.
- Experience with LLM orchestration frameworks (LangChain/LangGraph or equivalent), agentic patterns, tool use, and multi-agent workflows at an architecture level.
- Deep expertise in data and AI governance, data quality, lineage, security, and responsible AI - including PII handling, access control (Unity Catalog, RBAC), and audit.
- Proven ability to set architecture standards, lead design reviews, and provide technical leadership and mentorship across multiple engineering and data science teams.
- Strong communication and stakeholder-management skills; able to drive architecture decisions and translate business needs into technical blueprints for both technical and executive audiences.
Good to Have: Knowledge of LLMOps and observability tooling for AI systems; Databricks / cloud / AI certifications; experience leading cloud/data platform migration and modernization to Databricks; experience contributing to POCs, proposals, and RFPs.
AI GenAI Architecture
- Own the end-to-end architecture for AI/ML and GenAI solutions on Databricks - data and feature pipelines, model training/serving, RAG, and agentic applications.
- Design RAG and LLM architectures - retrieval and embedding pipelines, vector store design, prompt/evaluation strategy, and guardrails - using Mosaic AI and Foundation Model APIs.
- Define reference architectures, design patterns, and reusable frameworks for scalable, secure, and cost-optimized AI platforms.
Data Platform Foundation
- Architect the lakehouse foundation (Delta Lake, medallion, Unity Catalog) and reusable PySpark pipelines that feed AI and analytics workloads.
- Guide implementation using Databricks Workflows and Delta Live Tables, with standards for data quality, incremental loads, and schema evolution.
MLOps, Governance Reliability
- Define MLOps/LLMOps practices - model lifecycle, MLflow tracking, deployment, monitoring, and drift/quality evaluation in production.
- Establish AI and data governance, cataloging, lineage, access control, and responsible-AI standards (PII handling, guardrails, audit) ensuring security and compliance.
- Define observability, monitoring, and cost-governance practices for reliable and efficient AI systems at scale.
Technical Leadership Collaboration
- Set and enforce architecture standards and best practices; lead design and code reviews across multiple engineering and data science teams.
- Mentor engineers, ML engineers, and leads, and provide hands-on guidance from prototype to production readiness.
- Collaborate with product owners, engineering leaders, and business stakeholders to translate requirements into AI blueprints, POCs, and roadmaps.
Disclaimer: This job posting and location has been aggregated from external source. Role details, content, and availability are subject to change. Applicants are advised to confirm the latest information directly on the company website before applying.
Sourced from naukri
Skills
data qualityapisai servicesdata platformsamazon web servicesai governanceAnalyticsDelta Live TablesCloud DataDatabricksData EngineeringcloudArchitectureData GovernancecomplianceAi SolutionsMicrosoft AzureData ScienceArtificial Intelligence
About Impetus Technologies
Impetus Technologies
IT Services & Consulting
3.2k+
employees
1991
35 years old
Indore, Madhya Pradesh
India
About
We are a global digital engineering company focused on delivering leading-edge solutions and expert services to help enterprises achieve their transformation technology and business goals.
Industries
IT Services & ConsultingAnalytics & KPOEmerging TechnologiesSoftware Product
Employee ratings
654 reviews
3.4/ 5
Culture
3.3
Career growth
3.1
Work-life
3.3
Employees rate it well for
Work Life Balance137Work Culture85Learning85Salary76Senior Management62
Find them on