Senior Data Engineer(6+ years with MS Fabric, Azure, ADF, Data Platforms only)

Job Summary

We are looking for a Senior Data Engineer to design, build, and operate modern data solutions that power our data products, analytics, integrations, reporting, and emerging AI capabilities.

This is a hands-on engineering role for someone who combines strong data engineering fundamentals with experience in modern cloud data platforms, APIs, integration patterns, and production-grade engineering practices.

You will work across technologies such as Databricks, Microsoft Fabric, Snowflake, and cloud-native services, depending on the needs of the solution. We value engineers who understand the underlying architecture and engineering principles rather than being tied to a single technology.

We are also looking for an AI-first engineer who actively uses AI-assisted development to improve how solutions are designed, developed, tested, documented, and supported while maintaining strong engineering quality and accountability.

Key Responsibilities

Build Modern Data Solutions

• Design, develop, and maintain scalable data pipelines supporting operational, analytical, reporting, and AI use cases.
• Build batch, incremental, near-real-time, and event-driven data processing solutions.
• Develop data ingestion, transformation, orchestration, and delivery workflows using modern cloud data platforms.
• Design and implement data models optimized for performance, maintainability, and consumption.
• Develop reusable engineering components and patterns rather than one-off solutions.
• Optimize pipelines and data processing for reliability, scalability, performance, and cost.
• Contribute to technical design discussions and architecture decisions.

Build Enterprise Data Integrations

• Design and implement integrations across enterprise applications, data platforms, and third-party systems.
• Develop reusable integration services and data-access capabilities.
• Build secure REST and GraphQL APIs to expose data and platform capabilities.
• Implement event-driven and asynchronous integration patterns where appropriate.
• Integrate systems using APIs, connectors, messaging, events, and modern data-sharing techniques.
• Apply appropriate authentication, authorization, versioning, monitoring, and API lifecycle practices.

Engineer for Quality & Reliability

• Build automated data validation, reconciliation, and quality controls into pipelines and data products.
• Implement appropriate unit, integration, and data testing as part of development.
• Build monitoring and observability into pipelines, APIs, and integration services.
• Troubleshoot complex production issues and perform root-cause analysis.
• Implement preventive improvements rather than repeatedly addressing symptoms.
• Continuously improve the reliability, resiliency, maintainability, and performance of production solutions.

Apply AI-First Engineering Practices

• Use AI-assisted development throughout the engineering lifecycle, including design, coding, testing, documentation, troubleshooting, and optimization.
• Apply appropriate technical review and validation to AI-generated code and engineering artifacts.
• Practice spec-driven development by defining clear requirements, data contracts, API contracts, acceptance criteria, and test expectations before implementation.
• Use AI to accelerate engineering delivery while maintaining security, maintainability, and quality.
• Contribute to the development of data solutions supporting AI agents, RAG, semantic models, machine learning, and other AI-enabled applications.
• Experiment with emerging engineering capabilities and share effective practices with the broader team.

Enable Secure & Governed Data

• Implement data quality, lineage, metadata, security, and auditability requirements within data solutions.
• Build data pipelines and integrations appropriate for sensitive and regulated data.
• Follow established governance, risk, security, and compliance requirements.
• Support secure and auditable processing of KYC, UCC, public-record, and other business data.
• Maintain clear technical documentation for data flows, APIs, transformations, and operational processes.
• Partner with architecture, security, governance, and platform teams to resolve technical and compliance requirements.

Operate What You Build

• Take ownership of solutions through development, deployment, and production operation.
• Monitor pipeline execution, API health, data quality, and platform performance.
• Investigate incidents and integration failures and drive issues through resolution.
• Participate in code reviews and continuously improve engineering practices.
• Automate deployment, testing, configuration, and operational processes wherever practical.
• Contribute to improving team standards based on lessons learned from production.

Collaborate & Contribute

• Work closely with Lead Data Engineers, product teams, architects, analysts, AI teams, and business stakeholders.
• Translate requirements into practical and scalable technical solutions.
• Participate actively in architecture and technical design discussions.
• Communicate technical decisions, dependencies, risks, and tradeoffs clearly.
• Share knowledge and reusable solutions across the engineering team.
• Support and mentor less-experienced engineers when appropriate.
• Demonstrate ownership, curiosity, collaboration, and a continuous-learning mindset.

Required Qualifications

. Bachelor's degree in Computer Science, Engineering, Information Systems, or a related discipline, or equivalent practical experience.

. 6+ years of experience in data engineering, integration engineering, software engineering, or related roles.

. Strong SQL skills with experience in data modeling, database design, and performance optimization.

. Hands-on experience designing and developing scalable ETL/ELT pipelines and data-processing solutions.

. Experience with one or more modern cloud data platforms, such as Databricks, Microsoft Fabric, Snowflake, or comparable technologies.

. Programming experience with Python, PySpark, Scala, Java, C#, or a comparable language.

. Experience developing and consuming REST APIs and working with modern API-based architectures.

. Experience with event-driven, messaging, or asynchronous integration patterns.

. Experience with cloud-native technologies across Azure, AWS, GCP, or comparable environments.

. Experience with source control, automated testing, CI/CD, monitoring, and modern DevOps practices.

. Understanding of data quality, governance, security, lineage, and cloud identity concepts.

. Experience supporting production data systems and troubleshooting complex technical issues.

. Practical experience using AI-assisted development tools in day-to-day engineering.

. Strong problem-solving, communication, and collaboration skills.

Preferred Qualifications

. Experience working across multiple modern data platforms or cloud technologies.

. Experience with Apache Spark, Delta Lake, Apache Iceberg, lakehouse architectures, or comparable technologies.

. Experience with GraphQL and enterprise API management platforms.

. Experience with streaming and messaging technologies such as Kafka, Event Hubs, Event Grid, Service Bus, or comparable technologies.

. Experience with serverless computing and cloud-native integration services.

. Experience with data products and data-as-a-product principles.

. Experience with modern data-sharing and interoperability patterns.

. Experience with Infrastructure as Code and deployment automation.

. Experience with AI-related data solutions such as RAG, semantic models, data agents, machine learning, or agent/tool integrations.

. Experience working with sensitive or highly governed data in financial services, legal, compliance, or regulatory environments.

. Familiarity with concepts such as KYC, UCC, public records, GRC, or SOC 2.

Key Skills

Modern Data Platforms | Databricks | Microsoft Fabric | Snowflake | Cloud Data Services | SQL | Python / PySpark | Apache Spark | Data Modeling | ETL/ELT | REST & GraphQL APIs | Event-Driven Architecture | Data Integration | Data Quality & Observability | Data Governance & Lineage | CI/CD & DevOps | AI-Assisted Engineering | AI Agents & RAG

Our Interview Practices