Sr Staff Engineer

Responsibilities

• Distributed training at recommender scale. Our models are embedding-heavy - ID vocabularies in the 10 -10 range, sharded across workers and churning constantly. That makes training a different problem from most deep learning: embedding placement and sharding strategy, eviction, multi-host data-parallel throughput, and holding cost per experiment down across ~35 models in daily training.
• We run on TensorFlow today and are moving to the PyTorch ecosystem.
• Inference optimization across hardware and runtimes. Serving is the largest line in our infrastructure cost.
• We run TensorFlow Serving and OpenVINO today, across CPU and GPU; testing further runtimes - ONNX Runtime, Triton, TensorRT - is on the roadmap.
• The work is measurement-driven: find where cost and latency actually go, then move them.
• Expand the GPU serving path.
• We run a GPU/CPU hybrid deployment today, but only for a subset of models. Extending it - and making it the obvious choice wherever it wins on cost or unlocks models CPU cant serve - is open work.
• Orchestration and observability for dozens of training pipelines across feed and ads, running on Flyte and Kubernetes.
• Work horizontally across feed and ads, and raise the ceiling around you through design review and pairing.

Requirements

• 8+ years of professional software engineering experience - we care about the depth of what youve built more than the count. Fast progressors are very much welcome.
• Depth in at least one of: distributed training, model serving and inference optimization, or ML platform / MLOps - with working knowledge of the others.
• Were looking for T-shaped, not a tour of all three.
• Youve owned or substantially rebuilt a piece of production ML infrastructure, not only operated one someone else built. For example: changed how a training job was parallelized, moved a model between serving runtimes and measured the result, built eval and release gating for a model fleet, or debugged a distributed training failure to root cause.
• Comfortable in framework internals - PyTorch, TensorFlow, TorchRec, TF Serving / ONNX Runtime / Triton / OpenVINO - not only their APIs.
• You reason about performance from measurement: profiles, utilization, where the time actually goes.
• Strong software engineering fundamentals - distributed systems, system design, building for scale and reliability.
• Comfortable as a senior individual contributor, owning complex problems end to end with minimal hand-holding.
• We welcome candidates from software engineering, ML engineering, and ML platform backgrounds.
• We care what youve built, not which track you sat on.

Disclaimer: This job posting has been aggregated from external source. Role details, content, and availability are subject to change. Applicants are advised to confirm the latest information directly on the company website before applying.