Azure Data Engineer

Job Summary We are seeking an experienced

Azure Data Engineer ll Bangalore (hybrid) ll 7+ yrs of exp with strong expertise in Azure Databricks, PySpark, Python, and SQL to design, develop, and optimize scalable data solutions on the Azure cloud platform. The ideal candidate should have hands-on experience building ETL/ELT pipelines, implementing data migration projects, and developing modern data lakehouse architectures to support enterprise analytics and reporting needs.

About Us CodeVyasa is a mid-sized product engineering company that works with top-tier product and solutions organizations such as McKinsey, Walmart, RazorPay, Swiggy, and others. We are a team of 550+ engineers, driving innovation across Product & Data Engineering, focusing on Agentic AI, RPA, Full Stack, and GenAI-based solutions.

Primary Skills (Mandatory)

• Azure Cloud Services
• Azure Databricks
• PySpark / Apache Spark
• Python
• SQL
• Azure Data Factory (ADF)
• ETL / ELT Development
• Data Modeling and Data Transformation
• Experience working with Azure Data Lake Storage (ADLS Gen2)

Roles & Responsibilities

• Design, develop, and maintain scalable ETL/ELT data pipelines using Azure Data Factory, Databricks, and PySpark.
• Build and optimize data processing workflows for large-scale structured and unstructured datasets.
• Develop robust data ingestion frameworks from various source systems, databases, APIs, and external platforms.
• Implement and maintain data lakehouse architectures using Azure-native technologies.
• Perform data migration activities from on-premises or legacy systems to Azure cloud environments.
• Optimize Spark jobs and Databricks workloads for performance, scalability, and cost efficiency.
• Develop reusable frameworks, automation scripts, and data quality controls.
• Collaborate with business stakeholders, architects, and analytics teams to understand data requirements and deliver solutions.
• Implement monitoring, troubleshooting, and performance tuning for data pipelines.
• Follow best practices for security, governance, and compliance across data platforms.

Secondary Skills (Preferred)

• Delta Lake / Delta Live Tables (DLT)
• Azure Synapse Analytics
• Azure DevOps and CI/CD Pipelines
• Databricks Workflows / Apache Airflow
• Event-Driven Architecture
• Streaming Technologies (Kafka, Event Hub, Spark Streaming)
• API Integrations
• Data Warehousing and Lakehouse Architecture
• Data Governance and Data Quality Frameworks

Required Qualifications

• Bachelor's degree in Computer Science, Information Technology, Engineering, or a related field.
• Strong hands-on experience with Azure Data Engineering technologies.
• Proficiency in Python, SQL, and PySpark.
• Experience building enterprise-scale data pipelines and data integration solutions.
• Understanding of cloud-based data architectures and best practices.
• Experience working in Agile development environments.

Good to Have

• Azure Certifications (DP-203, Azure Data Engineer Associate, etc.)
• Experience with real-time data processing and streaming solutions.
• Exposure to modern analytics and reporting platforms.
• Knowledge of DevOps practices and Infrastructure as Code (IaC).

Why Join CodeVyasa?

• Work on innovative, high-impact projects with leading global clients.
• Exposure to modern technologies, scalable systems, and cloud-native architectures.
• Continuous learning and upskilling opportunities through internal and external programs.
• Supportive and collaborative work culture with flexible policies.
• Competitive salary and comprehensive benefits package.
• Free healthcare coverage for employees and dependents.