Rethinking Evidence Hierarchies in Medical Language Benchmarks: A Critical Evaluation of HealthBench

TL;DR

This paper critically evaluates the limitations of the HealthBench benchmark in medical language models, highlighting biases and regional disparities, and proposes a new framework grounded in clinical guidelines and systematic evidence to improve global relevance and trustworthiness.

Contribution

It introduces an evidence-based reward framework using clinical practice guidelines and systematic reviews to enhance the clinical validity and fairness of medical language model evaluation.

Findings

Identifies biases in current benchmarks due to expert opinion reliance.

Proposes a new reward system anchored in clinical guidelines and systematic reviews.

Aims to improve global relevance and ethical standards in medical AI evaluation.

Abstract

HealthBench, a benchmark designed to measure the capabilities of AI systems for health better (Arora et al., 2025), has advanced medical language model evaluation through physician-crafted dialogues and transparent rubrics. However, its reliance on expert opinion, rather than high-tier clinical evidence, risks codifying regional biases and individual clinician idiosyncrasies, further compounded by potential biases in automated grading systems. These limitations are particularly magnified in low- and middle-income settings, where issues like sparse neglected tropical disease coverage and region-specific guideline mismatches are prevalent. The unique challenges of the African context, including data scarcity, inadequate infrastructure, and nascent regulatory frameworks, underscore the urgent need for more globally relevant and equitable benchmarks. To address these shortcomings, we propose anchoring reward functions in version-controlled Clinical Practice Guidelines (CPGs) that incorporate systematic reviews and GRADE evidence ratings. Our roadmap outlines "evidence-robust" reinforcement learning via rubric-to-guideline linkage, evidence-weighted scoring, and contextual override logic, complemented by a focus on ethical considerations and the integration of delayed outcome feedback. By re-grounding rewards in rigorously vetted CPGs, while preserving HealthBench's transparency and physician engagement, we aim to foster medical language models that are not only linguistically polished but also clinically trustworthy, ethically sound, and globally relevant.