AutoRG-Brain: Grounded Report Generation for Brain MRI

TL;DR

AutoRG-Brain is a novel AI system that generates detailed, grounded reports for brain MRI scans, improving radiologists' efficiency and accuracy through pixel-level visual clues and comprehensive datasets.

Contribution

This paper introduces the first brain MRI report generation system with pixel-level grounded visual explanations and releases a new dataset to support AI-assisted radiology research.

Findings

System enhances junior doctors' report accuracy

Improves alignment with senior radiologists' performance

Demonstrates reliability through quantitative and human evaluations

Abstract

Radiologists are tasked with interpreting a large number of images in a daily base, with the responsibility of generating corresponding reports. This demanding workload elevates the risk of human error, potentially leading to treatment delays, increased healthcare costs, revenue loss, and operational inefficiencies. To address these challenges, we initiate a series of work on grounded Automatic Report Generation (AutoRG), starting from the brain MRI interpretation system, which supports the delineation of brain structures, the localization of anomalies, and the generation of well-organized findings. We make contributions from the following aspects, first, on dataset construction, we release a comprehensive dataset encompassing segmentation masks of anomaly regions and manually authored reports, termed as RadGenome-Brain MRI. This data resource is intended to catalyze ongoing research and development in the field of AI-assisted report generation systems. Second, on system design, we propose AutoRG-Brain, the first brain MRI report generation system with pixel-level grounded visual clues. Third, for evaluation, we conduct quantitative assessments and human evaluations of brain structure segmentation, anomaly localization, and report generation tasks to provide evidence of its reliability and accuracy. This system has been integrated into real clinical scenarios, where radiologists were instructed to write reports based on our generated findings and anomaly segmentation masks. The results demonstrate that our system enhances the report-writing skills of junior doctors, aligning their performance more closely with senior doctors, thereby boosting overall productivity.