Azure Databricks Solution Architect

Job Description:

• Infosys is seeking an experienced Azure Databricks Solution Architect to lead the design and implementation of modern cloud based data platforms leveraging Azure Databricks PySpark and Big Data technologies
• The ideal candidate will have extensive experience in architecting scalable Data Engineering solutions defining enterprise data strategies implementing Lakehouse Architecture and driving cloud transformation programs using Azure Databricks and the broader Azure Data Platform ecosystem

Key Responsibilities:

• Lead architecture design and implementation of enterprise scale Azure Databricks solutions
• Define and govern modern Data Engineering and Data Architecture standards
• Design scalable ETL ELT frameworks data pipelines and cloud native data platforms
• Architect solutions using Azure Databricks PySpark Azure Data Factory ADLS Gen2 and Azure Synapse Analytics
• Build and optimize large scale Data Lake Delta Lake and Lakehouse Architecture implementations
• Drive cloud migration and modernization initiatives from legacy platforms to Azure
• Collaborate with business leaders enterprise architects and delivery teams to define solution roadmaps
• Lead performance optimization scalability improvements and platform modernization efforts
• Establish best practices around Data Governance Security Data Quality and Compliance
• Support project estimation solution reviews architecture governance and technical presales activities
• Mentor Data Engineers and technical teams on Azure and Databricks best practices
• Evaluate emerging Azure and Databricks capabilities to enable innovation and platform optimization
• Required Skills
• 12 14 years of experience in Data Engineering Data Architecture and Cloud Data Platforms
• Strong expertise in Azure Databricks and enterprise scale Databricks implementations
• Deep hands on experience with PySpark Apache Spark Spark SQL Python and SQL
• Extensive knowledge of Azure Data Factory ADF Azure Data Lake Storage ADLS Gen2 and Azure ecosystem services
• Proven experience designing and implementing Lakehouse Architecture and Delta Lake solutions

Technical Requirements:

• Experience with Azure Synapse Analytics and enterprise analytics platforms
• Exposure to Structured Streaming Real Time Data Processing and Event Driven Architectures
• Knowledge of Medallion Architecture and modern Data Mesh concepts
• Experience with multi cloud environments including AWS and GCP
• Familiarity with Infrastructure as Code Terraform ARM Templates
• Experience leading cloud migration and digital transformation programs

Preferred Skills:

Technology->Data Engineering->Databricks,Technology->Big Data - Data Processing->PySpark,Technology->Cloud Platform->Azure Networking Services->AzureCDN