Azure Data Engineer

Greetings from Tata Consultancy Services (TCS)!

We are seeking experienced Azure Data Engineers with strong expertise in Azure-based data platforms, big data technologies, and modern data engineering practices.

Experience: 5 to 10 Years

Location: Chennai

Technical Skills Required

• Strong experience in Azure Data Lake, Azure Databricks, and Azure Data Factory (ADF).
• Hands-on expertise in SQL, PySpark, or Scala for building scalable ETL/ELT solutions.
• Experience with batch and real-time data ingestion frameworks, including Kafka.
• Strong knowledge of large-scale data processing and big data architectures.
• Experience with relational and NoSQL databases such as SQL Server, Oracle, Azure SQL, and Cosmos DB.
• Hands-on experience with Informatica PowerCenter, Informatica IDMC, or equivalent cloud data integration tools.
• Experience with Azure DevOps (AzDO), GitHub, and CI/CD pipeline implementation.
• Practical usage of Microsoft Copilot to improve development productivity and data engineering workflows.
• Exposure to Generative AI (GenAI) use cases in data platforms and analytics solutions.
• Familiarity with enterprise scheduling tools such as Maestro and data management solutions such as Precisely.
• Domain knowledge in the Insurance industry, including Policy, Claims, Billing, Underwriting, Actuarial, and Regulatory data processes.

Roles & Responsibilities

• Design, develop, and maintain scalable data pipelines using Azure Data Lake, Databricks, and ADF.
• Build robust ETL/ELT frameworks for batch and real-time data processing.
• Develop data transformation logic using PySpark, Scala, and SQL.
• Implement and optimize data ingestion from multiple structured and unstructured data sources.
• Work with Kafka-based streaming solutions and large-scale distributed data processing systems.
• Develop and maintain CI/CD pipelines using Azure DevOps and GitHub.
• Integrate and manage cloud-based data integration solutions using Informatica platforms.
• Implement data quality, governance, security, and performance optimization best practices.
• Collaborate with business stakeholders, architects, and development teams to deliver data-driven solutions.
• Explore and implement GenAI and Copilot-enabled solutions to improve engineering efficiency and analytics capabilities.
• Support insurance data modernization initiatives and regulatory reporting requirements.

Preferred Skills

• Insurance domain experience.
• Strong analytical and problem-solving skills.
• Experience working in Agile environments.
• Excellent communication and stakeholder management capabilities.

Interested candidates can share their updated CV.