Spatial Reasoning Evaluation Specialist (VIDEO RL)
Careerblock · Noida · onsite · Posted 46d ago
Not disclosedFresherfulltimeSourced from
About a minute. You'll make an account on the way through.
About this role
About The Opportunity
A fast-scaling AI research and evaluation lab in the Generative AI & Spatial Reasoning domain, we build and benchmark next-gen video-based reinforcement learning (RL) agents that perceive, plan, and act in 3D environments. Our work powers real-world embodied AI applications—from robotics navigation to autonomous systems—by rigorously testing how models understand spatial relationships, motion logic, and scene dynamics from video input. We're hiring a Spatial Reasoning Evaluation Specialist to design, execute, and scale high-fidelity human-in-the-loop evaluations for video RL models.
Role & Responsibilities
A fast-scaling AI research and evaluation lab in the Generative AI & Spatial Reasoning domain, we build and benchmark next-gen video-based reinforcement learning (RL) agents that perceive, plan, and act in 3D environments. Our work powers real-world embodied AI applications—from robotics navigation to autonomous systems—by rigorously testing how models understand spatial relationships, motion logic, and scene dynamics from video input. We're hiring a Spatial Reasoning Evaluation Specialist to design, execute, and scale high-fidelity human-in-the-loop evaluations for video RL models.
Role & Responsibilities
- Design and implement standardized evaluation protocols to assess spatial reasoning performance of video RL agents across tasks like path planning, object interaction, and scene navigation.
- Annotate, curate, and label video datasets to create ground-truth benchmarks for agent evaluation, with emphasis on temporal-spatial consistency and action causality.
- Run human-in-the-loop experiments comparing agent output against human baseline performance using structured scoring rubrics and UI tools.
- Identify failure modes, edge cases, and spatial reasoning gaps in agent behavior and document findings for research and engineering teams.
- Collaborate with ML researchers to iterate on evaluation metrics, refine task design, and improve benchmark robustness.
- Build and maintain evaluation dashboards to track agent performance trends, score distributions, and spatial reasoning drift over time.
- Must-Have
- PyTorch
- OpenCV
- Video annotation tools (e.g., CVAT, Labelbox)
- Spatial reasoning task design
- Human-in-the-loop evaluation frameworks
- Reinforcement learning basics (e.g., PPO, DQN, env interaction)
- Python scripting for data validation & reporting
- 3D scene understanding (e.g., coordinate frames, depth maps, camera poses)
- Preferred
- Experience with embodied AI platforms (e.g., Habitat, AI2 Thor, Unity ML-Agents)
- Familiarity with RLlib or RL frameworks for agent benchmarking
- Background in cognitive science or psychophysics of spatial reasoning
- Work directly with cutting-edge video RL models shaping the future of embodied AI.
- On-site innovation lab environment in India with access to high-end annotation tools and compute infrastructure.
- Opportunity to co-author research evaluations and contribute to public benchmarks in spatial reasoning.
Sourced from foundit
Skills
video annotation toolsHuman-in-the-loop evaluation frameworksPython scripting for data validation reportingReinforcement learning basicsSpatial reasoning task designOpencvPytorch3D scene understanding