Data Engineer

JD:

The Skills That Are Key to This Role

• Translate complex business and analytical needs into scalable, resilient data architectures and pipelines.
• Design and implement highperformance distributed pipelines (batch & streaming) with strong SLAs, monitoring, and cost efficiency.
• Build lakehouse and warehouse solutions (e.g., Delta/Iceberg/Hudi + Snowflake) with robust semantic/data models.
• Endtoend ownership: requirements architecture build deploy monitor optimize.
• ETL/ELT excellence with Informatica, SnapLogic, Azure Data Factory, Azure Functions, Python, and Databricks/Spark.
• Azure platform proficiency, including:
• Azure Data Factory, Azure Databricks, AKS, Azure Service Bus (ASB), API Management, Storage Accounts, Event Hub/Event Grid, Redis Cache.
• Strong RDBMS/NoSQL experience: Oracle, SQL Server/SQL MI, MySQL, Postgres, Cosmos DB, MongoDB.
• DataOps/DevOps: CI/CD pipelines (GitHub Actions/Azure DevOps), unit/integration tests, automated data quality, and infrastructureascode (Bicep/ARM/Terraform).
• Data quality & observability (e.g., Great Expectations/Deequ/Monte Carlo), schema evolution, partitioning, and performance tuning.
• Build tools and frameworks that turn pipelines into actionable insights for key business KPIs; partner with BI/analytics teams.
• Strong Agile deliverydriving epics, stories, and tasks using Agile/Scrum and Jira; excellent written and verbal communication.