Lead Data Engineer

Senior Data Platform Engineer / Data Architect. Every number a customer argues about. Architecture is still open for you to decide and defend: warehouse shape, table format, orchestration, where the streaming boundary sits against change capture or a tighter batch cadence, and how datasets are tiered with real service levels behind them. This is not an analytics engineer role. They decide what a term means; you decide whether the number is computed correctly, arrives on time, and can bear the inference drawn from it.

Responsibilities:

• Own the event pipeline behind shipment milestones, late arrivals, corrections that restate a milestone recorded three days ago, and the gap between event time and ingest time. Define how a restatement propagates to something a customer has already seen.
• Make the serving path good enough for an agent to depend on to define a freshness target for every dataset and know the p99 latency of the read path, because when a run stalls waiting for a number, a workflow with money attached stalls with it.
• Build data quality as failing checks, not dashboards with null rates, referential breaks, duplicate milestones, and distribution shifts when a partner changes a field. A check that only warns is a check nobody reads.
• Enforce tenant isolation at the data layer and assert it through automated tests. Contract rates are commercially sensitive when competitors share the same forwarders.
• Plan backfills that cost real money; reprocessing across 30+ countries of historical data has both a financial cost and a blast radius. Design the process accordingly.
• Bring statistical judgment where decisions need it to understand what observational data can and cannot prove, and say so before the decision is made rather than after.

Requirements:

• Years are a floor, not the bar. If you miss one line but are exceptionally strong on the rest, apply and tell us which one.
• 5+ years building and operating production data platforms, including at least 2 years as the person accountable for architecture that other engineers and teams depended on.
• You have designed a warehouse from first principles, using grain, dimensional models, slowly changing dimensions, and additive vs. non-additive measures, and you can describe a modelling trap you found in someone else's work and fixed.
• Deep SQL and Python window functions, incremental and idempotent transformations, safe-to-rerun backfills, and pipelines that survive late-arriving data.
• Hands-on ownership of a production orchestrator like Airflow, Dagster, Prefect, Temporal, or equivalent, including retries, service levels, dependency modelling, and backfill strategy.
• A defensible position on table formats and streaming vs. change capture vs. tighter batch cadence, with the reasoning behind the choice rather than following fashion.
• With genuine statistical rigor, you can design an experiment, defend the power calculation, and choose the right statistical test for skewed or correlated data.
• You have led engineers hiring, code/design review, and mentoring and have carried a pager for datasets that other teams depended on.

Also Good: None of this is required:

• You have run a data on-call rotation and can describe what you changed to make it quieter.
• You have worked in a domain where being wrong about a time or quantity had a cost someone could actually name.