Sr. Data Engineer

As a Senior Data Engineer / Databricks specialist, you will design and build enterprise-grade data and AI solutions that power advanced analytics and Agentic AI use cases across Zurich.

Responsibilities:

• Design and build scalable solutions to ingest, store and process very large amounts of data (structured, semi-structured, and unstructured), including streaming and real-time, into the cloud-based Lakehouse.
• Build enterprise-grade, highly reusable data assets for analytics consumption.
• Develop conceptual, logical, and physical data architecture blueprints and system specifications to develop a single source of truth for the enterprise.
• Partner with analytics experts, data scientists, and Business Unit analysts to build self-service experiences in our data democratization platform.
• Take ownership of the operationalisation of Agentic AI and ML models while designing solutions to optimise data pipelines, model deployment, and model monitoring.

AI related expectations:

• Use AI-enabled tools and platforms, where available, across the full Data and AI engineering lifecycle (for example, for code generation, testing, documentation, pipeline monitoring, and incident analysis), while maintaining full accountability for the quality and robustness of solutions.
• Identify and propose opportunities to redesign workflows and data processes using automation, AI agents and generative AI, with a focus on reducing manual work and improving reliability and time to value.
• Collaborate with data scientists, architects and business stakeholders to ensure Data and AI solutions are secure, well governed and aligned with Zurich's responsible AI principles.

Requirements:

• Experience with Azure or AWS Cloud.
• Experience in building enterprise-grade Data and AI solutions.
• Experience with various Microsoft Azure services (including Fabric, Databricks, Data Factory, Cosmos DB, Document Intelligence, Cognitive Services, AI Foundry, Power Platform, Copilot Studio, Azure ML, Power Automate, Power BI, etc. ).
• Experience with vector stores like Elastic, Pinecone, Mongo Atlas, or Redis.
• Experience with cloud data warehouse solutions like Snowflake, Fabric, or Redshift.
• Exposure to ML frameworks including TensorFlow, PyTorch, Scikit-learn.
• Exposure to deployments for LLMs including end-to-end unstructured data pipeline build, design of vector stores, embeddings, and prompt engineering.