Databricks Forward Deployed Engineer

Job Title: Databricks Forward Deployed Engineer (FDE)

Experience: 58 Years
Location: PAN India
Open Positions: 10
Function: Data & AI Engineering
Level: Senior (G7)

Job Description

We are looking for experienced Databricks Forward Deployed Engineers (FDEs) with strong expertise in data engineering, Apache Spark, PySpark, Azure, CI/CD, and exposure to Generative AI. The ideal candidate will work closely with enterprise clients to design, develop, optimize, and deploy scalable, production-grade data platforms on Databricks.

The role requires strong technical problem-solving, solution design, performance optimization, and client-facing communication skills.

Key Responsibilities

• Design and implement scalable Lakehouse architectures using Databricks and Medallion Architecture (Bronze, Silver, Gold).
• Develop and optimize production-grade data pipelines using PySpark, Spark SQL, and Delta Lake.
• Implement batch and streaming data ingestion using Auto Loader, Kafka, and event-driven architectures.
• Implement CDC and SCD Type 1, Type 2, and Type 3 patterns.
• Optimize Spark jobs, Delta Lake tables, cluster configurations, and overall platform performance.
• Configure Unity Catalog, data governance, access controls, and data lineage.
• Build and manage CI/CD pipelines using Databricks Asset Bundles (DABs), GitHub Actions, or Azure DevOps.
• Troubleshoot Spark performance issues, including data skew, shuffle overhead, and memory-related errors.
• Support GenAI solutions involving RAG pipelines, vector databases, embeddings, and LLM evaluation.
• Engage with clients to understand business requirements, recommend technical solutions, and communicate architecture decisions.
• Establish data quality checks, monitoring, production alerting, and SLA frameworks.

Mandatory Skills

• Python and PySpark
• Apache Spark and Spark SQL
• Databricks and Delta Lake
• Microsoft Azure
• React
• Generative AI (GenAI)
• CI/CD and deployment automation
• Strong SQL and data pipeline development

Required Qualifications

• 5+ years of experience in data engineering.
• 2+ years of hands-on experience with Databricks or Apache Spark.
• Strong knowledge of PySpark, Spark SQL, Delta Lake, and performance tuning.
• Experience implementing enterprise Lakehouse and Medallion architectures.
• Hands-on experience with CI/CD pipelines and multi-environment deployments.
• Understanding of data governance, access controls, and production monitoring.
• Excellent communication, analytical thinking, and stakeholder management skills.
• Prior experience in client-facing, consulting, or enterprise delivery environments.

Good to Have

• Databricks Certified Data Engineer Associate or Professional certification.
• Experience with Structured Streaming, Delta Live Tables, or Kafka.
• Exposure to Mosaic AI, Databricks Genie, RAG, vector stores, and LLM evaluation.
• Experience with Azure DevOps, GitHub Actions, or Databricks Asset Bundles.
• Exposure to AWS or GCP cloud platforms.
• Experience in consulting, GSI, or enterprise client delivery environments.

If interested , please apply using link below :

https://www.intervyo.ai/apply/ce8e8d8b-a8fe-41df-ac5c-d06780f123cf

or email at [email removed]

Thanks

Team HR