Databricks architect

Job Description

Overview

We are seeking an experienced Data & AI Architect to lead the design and delivery of modern, AI-first data solutions using Azure Databricks, Lakehouse, Lakebase, Databricks Genie, and Azure AI services. The role combines enterprise data architecture, hands-on technical leadership, AI-first engineering, solutioning, and delivery leadership across complex data modernization initiatives.

Responsibilities

AI-First Delivery Leadership

• Embed AI-first mindset into delivery workflows, leveraging automation and intelligent orchestration where applicable.
• Lead end-to-end delivery of complex Databricks Lakehouse, AI, and data modernization projects.
• Drive engineering excellence through reusable components, accelerators, and scalable Azure Databricks Lakehouse architecture.
• Act as a technical point of contact for clients, translating business requirements into scalable Azure, Databricks, Lakehouse, and AI solutions.
• Lead technical solutioning, estimations, and support architecture design during pre-sales and delivery shaping.

Qualifications

Qualifications & Preferred Skills

• 12+ years of experience in software/solution engineering, with at least 3-4 years as Architect and Delivery leadership roles.
• Excellent communication, stakeholder management, problem-solving, and team leadership skills.
• Bachelors degree in computer science, Engineering, or related field (or equivalent experience).
• Relevant certifications are a plus.

Areas of Expertise

Enterprise Data Architecture & Modern Platforms

• Drive and influence architectural decisions for modern Lakehouse platforms, providing hands-on delivery leadership across Azure Databricks and Azure.
• Design and implement scalable Databricks Lakehouse architectures supporting enterprise data engineering, analytics, AI, and governed self-service.
• Lead architecture and adoption of Databricks Lakebase for operational and application-oriented data workloads where applicable.

Data Engineering & Large Scale Data Processing

• Hands-on leadership in designing, building, and operating large-scale batch and streaming data pipelines using Apache Spark, Azure Databricks, Delta Lake, and Azure.
• Lead and perform performance assessments, compute sizing, partition strategies and concurrency management.
• Drive implementation of scalable Lakehouse medallion architectures, data pipelines, and governed data products using Databricks and Azure.

AI First Data Engineering & Unify Your Data

• Design and implement agentic AI workflows that assist in data discovery, preparation, profiling, validation, and performance optimization.
• Actively adopt and promote use large language models to automate data engineering tasks such as schema inference, pipeline generation, metadata enrichment, documentation, and tuning recommendations.
• Lead AI powered data wrangling solutions, while maintaining governance, explainability, and human in the loop controls.

AI & Advanced Analytics Enablement

• Design, lead to implement, and optimize solutions using AI Agents where viable using Databricks Genie, Azure AI services, and Databricks Lakehouse capabilities.
• Lead adoption of Databricks Genie to enable governed natural-language access to enterprise data and analytics.
• Ensure AI and analytics solutions are grounded help improving trust, explainability, and relevance of AI outputs.
• Balance governance, performance, and usability by implementing governance by default patterns that scale self-service access without introducing friction.

Solutioning, Pre-Sales & Technical Leadership

• Lead technical solutioning, estimations, and support architecture design during pre-sales and delivery shaping.
• Translate business outcomes into clear Azure Databricks, Lakehouse, Lakebase, Genie, and AI architecture, execution plans, and risk mitigation strategies.
• Act as a trusted advisor to clients and internal leadership.
• Role & responsibilities

Preferred candidate profile