Data Architect

About the Role

Lead the architecture, design and delivery of enterprise-scale Databricks data platforms across Azure, AWS or GCP. The role will provide technical leadership for modern data engineering, Lakehouse architecture, data integration, data migration, analytics and AI/ML workloads while ensuring scalable, secure, performant and cost-effective solutions.

Responsibilities

• Design and implement enterprise-scale Databricks Lakehouse architectures.
• Define architecture across Azure, AWS and GCP based on business and technical requirements.
• Design data ingestion, transformation and processing using Databricks, Spark, PySpark, SQL and Python.
• Establish Lakehouse architecture, including Delta Lake, Medallion Architecture and data modelling.
• Design batch and real-time data processing and integration solutions.
• Architect and implement Databricks Workflows, Jobs, Unity Catalog and governance capabilities.
• Define data security, access control, lineage, quality, monitoring and governance standards.
• Design scalable and optimized solutions for performance and cloud cost.
• Lead migration and modernization of legacy data platforms to Databricks.
• Integrate Databricks with cloud-native services across Azure / AWS / GCP.
• Establish CI/CD, Git, DevOps and deployment standards for data engineering.
• Provide technical leadership through design reviews, code reviews and engineering standards.
• Guide and mentor Data Engineers and technical teams.
• Conduct POCs and evaluate new Databricks/cloud capabilities.
• Work with business stakeholders, project managers and enterprise architects to translate requirements into technical solutions.
• Support estimation, solution proposals, technical presentations and client discussions.
• Adhere to the Information Security Management policies and procedures.

Required Skills

• 10+ years of experience in Data Engineering / Data Architecture.
• Strong hands-on experience with Databricks and Lakehouse architecture.
• Strong knowledge of Apache Spark, PySpark, Python and SQL.
• Experience with Delta Lake and Medallion Architecture.
• Strong experience in data modelling, ETL/ELT and data integration.
• Experience designing scalable batch and real-time data solutions.
• Strong knowledge of at least one major cloud platform: Azure, AWS or GCP.
• Understanding of cloud data services and integration patterns.
• Experience with Databricks Workflows/Jobs and Unity Catalog.
• Strong understanding of data security, governance, quality and lineage.
• Experience with CI/CD, Git and DevOps practices.
• Strong solution architecture and technical leadership skills.

Preferred Skills /Good to Have

• Snowflake
• Azure Data Factory / AWS Glue / GCP Dataflow
• Kafka / Event Hubs / Kinesis / Pub/Sub
• Databricks ML / MLflow