RADAR: A Multimodal Benchmark for 3D Image-Based Radiology Report Review

TL;DR

RADAR introduces a comprehensive multimodal benchmark dataset for analyzing discrepancies in radiology reports by pairing 3D medical images with reports and edits, facilitating the development of models that support clinical review and quality assurance.

Contribution

This work presents RADAR, a novel benchmark dataset with structured discrepancy assessment tasks for multimodal models in radiology report review, emphasizing clinical reasoning and image-text alignment.

Findings

Provides expert-annotated abdominal CT dataset

Defines structured tasks for discrepancy evaluation

Supports systematic comparison of multimodal review models

Abstract

Radiology reports for the same patient examination may contain clinically meaningful discrepancies arising from interpretation differences, reporting variability, or evolving assessments. Systematic analysis of such discrepancies is important for quality assurance, clinical decision support, and multimodal model development, yet remains limited by the lack of standardized benchmarks. We present RADAR, a multimodal benchmark for radiology report discrepancy analysis that pairs 3D medical images with a preliminary report and corresponding candidate edits for the same study. The dataset reflects a standard clinical workflow in which trainee radiologists author preliminary reports that are subsequently reviewed and revised by attending radiologists. RADAR defines a structured discrepancy assessment task requiring models to evaluate proposed edits by determining image-level agreement, assessing clinical severity, and classifying edit type (correction, addition, or clarification). In contrast to prior work emphasizing binary error detection or comparison against fully independent reference reports, RADAR targets fine-grained clinical reasoning and image-text alignment at the report review stage. The benchmark consists of expert-annotated abdominal CT examinations and is accompanied by standardized evaluation protocols to support systematic comparison of multimodal models. RADAR provides a clinically grounded testbed for evaluating multimodal systems as reviewers of radiology report edits.