Data Engineer Advisor

Job Duties: ETL Engineering & Data Pipeline Development

Design, build, and maintain Azurebased ETL pipelines (e.g., Data Factory, Databricks, Data Lake) to ingest, clean, transform, and aggregate compensationrelated datasets across multiple regions.
Engineer upstream processes to produce 910 monthly aggregated output files (customer, revenue, product, sales rep, etc.), delivered 3 per month.
Ensure repeatability, monitoring, orchestration, and errorhandling for all ingestion and transformation workflows.Contribute to the creation of a master stitched data file to replace Varicents current dataassembly functions.

Business Rules Engine Development

Build, configure, and maintain a rules engine (ODM, Drools, or similar) to externalize business logic previously embedded in code.
Translate rules and logic captured by analysts and business SMEs into scalable, testable engine components.
Implement versioning, governance, and validation mechanisms for all logic used in compensation calculations.Ensure rule changes can be managed safely, reducing risk in highstakes compensation scenarios.

Data Modeling & Modern Architecture Implementation

Partner with data architects to implement the targetstate Azure data architecture for compensation analytics.
Develop optimized, scalable physical data models aligned to business logic and downstream needs.
Integrate with MDM sources and temporary EU workarounds, helping unify regional variations into a consolidated model.Build reusable, parameterized data pipelines and frameworks supporting longterm extensibility.

CrossFunctional Collaboration

Work closely with business analysts, data analysts, architects, and product owners across NA and Europe.
Participate in data discovery sessions, helping interpret and validate logic, rules, and data patterns.
Support three Scrum teams delivering compensation modernization, ensuring clarity on transformations and dependencies.Collaborate with QA, data quality testers, and governance teams to enforce validation standards.

Quality, Performance & Reliability

Implement data quality checks, profiling, reconciliation, and alerting across ingestion and transformation pipelines.
Engineer performanceoptimized pipelines capable of processing large, complex datasets multiple times per month.
Ensure compliance with audit, traceability, and business continuity expectations for compensation data.

Qualifications

58+ years of experience as a Data Engineer with strong handson expertise in Azure (Data Factory, Databricks, Data Lake Storage, SQL, Synapse preferred).
Proven ability to build productiongrade ETL/ELT pipelines supporting complex, multiregional business processes.
Experience designing or implementing rules engines (Drools, ODM, or similar).Strong SQL skills and experience with data modeling, data orchestration, and pipeline optimization.
Experience working in Agile Scrum teams and collaborating across global regions (U.S. and India preferred).
Ability to partner closely with analysts and business stakeholders to translate rules into technical solutions.
Excellent debugging, optimization, and engineering problemsolving skills.

Minimum Skills Required: SQL, Python, Azure Data Factory, Databricks, Azure Synapse