Today, we’re introducing HealthBench: a new benchmark designed to better measure capabilities of AI systems for health. Built in partnership with 262 physicians who have practiced in 60 countries, HealthBench includes 5,000 realistic health conversations, each with a custom physician-created rubric to grade model responses.

HealthBench is grounded in our belief that evaluations for AI systems in health should be:

Alongside the HealthBench benchmark, we're also sharing how several of our models perform, setting a new baseline to improve upon.

HealthBench tests how well AI models perform in realistic health scenarios, based on what physician experts say matters most.