Software Engineer II

Summary We are seeking a

Senior Data Engineer to architect and build modern Azure-based data solutions, lead large-scale data platform modernization initiatives, and mentor junior engineers. The ideal candidate should have deep expertise in

Azure Synapse Analytics, Databricks, Azure Data Factory (ADF), and modern data engineering practices . Experience with

Informatica PowerCenter is a plus but not mandatory .

Key Responsibilities

• Design and implement end-to-end Azure data architectures using Azure Synapse Analytics, Databricks, and ADLS Gen2.
• Lead and support data modernization and migration initiatives to Azure.
• Perform solution assessment, complexity analysis, and migration planning.
• Develop scalable data pipelines using PySpark, Spark SQL, and Azure Data Factory (ADF).
• Implement data engineering best practices including SCD, CDC, UPSERT/MERGE, and data quality frameworks.
• Optimize data processing performance, storage design, and cloud costs.
• Design partitioning, data distribution, and performance tuning strategies.
• Mentor team members and establish coding standards and engineering best practices.
• Drive adoption of AI-assisted development tools such as GitHub Copilot to improve productivity and code quality.

Required Skills

• Strong hands-on experience with: + Azure Synapse Analytics + Azure Databricks + PySpark + Spark SQL + Azure Data Factory (ADF)
• Advanced SQL expertise across platforms such as: + Oracle + SQL Server + PostgreSQL
• Experience with Git/GitHub, CI/CD, and Azure DevOps.
• Strong understanding of: + Data Warehouse Concepts + Medallion Architecture + Data Governance + Data Security (RBAC, Key Vault)
• Experience designing scalable and enterprise-grade data platforms.

Good to Have

• Exposure to Informatica PowerCenter (IPC).
• Experience with Kafka, Azure Event Hubs, or Streaming Architectures.
• Terraform or Infrastructure as Code (IaC).
• Power BI and Analytics solutions.
• AI/ML integration with data platforms.
• Domain experience in Insurance or Financial Services.