Medical AI Evaluation & Annotation

Theora works with leading healthcare organizations and frontier AI labs to evaluate how AI models perform on real-world radiology tasks . We're looking for practicing radiologists to help build and validate high-quality radiology evaluation data using clinical context, imaging studies, radiology reports, and downstream clinical evidence.

• Review radiology cases across CT, MRI, X-ray, ultrasound, and other imaging modalities
• Interpret imaging studies using relevant clinical history and prior imaging
• Annotate findings including diagnosis, localization, measurements, severity, and interval change
• Create gold-standard interpretations and reference answers
• Evaluate AI-generated radiology outputs for clinical accuracy, completeness, and safety
• Grade model reasoning using structured clinical rubrics
• Identify model failure modes such as missed findings, incorrect localization, poor comparison with priors, or inappropriate recommendations

Image interpretation Review a CT study and determine whether the model correctly identifies and localizes clinically significant abnormalities.

Longitudinal reasoning Compare current and prior imaging and determine whether the model correctly identifies progression, stability, or resolution.

Clinical-context reasoning Determine whether the model appropriately incorporates the imaging indication, relevant history, and prior studies into its interpretation.

• MD / equivalent medical degree
• Completed or currently completing Radiology training
• Board-certified / board-eligible Radiologist preferred
• Strong experience interpreting diagnostic imaging
• Comfortable with CT, MRI, X-ray, ultrasound, or relevant subspecialty imaging
• Excellent attention to detail
• Strong written clinical communication
• Ability to explain clearly why an interpretation is correct or incorrect

Especially interested in candidates with:

• Subspecialty fellowship training (e.g., neuroradiology, MSK, body/oncologic imaging, thoracic, breast)
• Prior experience with AI/CAD tools in clinical workflow
• Prior research, peer review, or teaching experience (residents/fellows) — useful for producing rubric-quality gold-standard reasoning, not just answers

• email [HIDDEN TEXT] with your resume!