Resident Solution Architect

Key Responsibilities

• Design and architect scalable data engineering and analytics solutions using Databricks.
• Work closely with clients to understand business requirements and translate them into technical solutions.
• Lead the architecture and delivery of complex Databricks projects from design through production.
• Develop and optimize data pipelines using Apache Spark, PySpark, SQL, and Databricks.
• Apply Spark performance optimization techniques and understand Spark runtime internals.
• Design solutions across cloud platforms and recommend appropriate cloud architecture.
• Support CI/CD and production deployment of data engineering workloads.
• Provide technical leadership and guidance to engineering teams.
• Advise clients on Databricks platform capabilities, architecture, scalability, security, and best practices.
• Support MLOps initiatives and integration of machine learning workloads with data platforms.
• Troubleshoot complex performance, scalability, and production issues.
• Stay current with the latest Databricks products, features, and platform capabilities.

Required Skills & Experience

• 7+ years of experience in Data Engineering, Data Platforms, and Analytics.
• 10+ years of consulting experience.
• Hands-on delivery experience in 68+ Databricks projects.
• Strong expertise in Databricks, Apache Spark, PySpark, and SQL.
• Deep understanding of distributed computing and Spark architecture/runtime internals.
• Working knowledge of at least two cloud platforms: AWS, Azure, or GCP.
• Deep expertise in at least one major cloud ecosystem.
• Experience with CI/CD and production deployments.
• Working knowledge of MLOps.
• Strong understanding of Databricks Lakehouse architecture and platform capabilities.
• Experience with performance tuning, optimization, and scalability.
• Strong client-facing communication, consulting, and solution-architecture skills.

Certification

• Databricks Certified Data Engineer Professional certification required/preferred.