Towards Interpretable Radiology Report Generation via Concept Bottlenecks using a Multi-Agentic RAG

TL;DR

This paper proposes an interpretable framework for radiology report generation from chest X-rays using concept bottleneck models and a multi-agent retrieval-augmented generation system, improving explainability and clinical utility.

Contribution

It introduces a novel multi-agent RAG system guided by concept bottleneck models for interpretable report generation from medical images.

Findings

Achieved 81% classification accuracy on COVID-QU dataset.

Generated reports with 84-90% scores on key metrics.

Enhanced interpretability and clinical relevance of AI-generated reports.

Abstract

Deep learning has advanced medical image classification, but interpretability challenges hinder its clinical adoption. This study enhances interpretability in Chest X-ray (CXR) classification by using concept bottleneck models (CBMs) and a multi-agent Retrieval-Augmented Generation (RAG) system for report generation. By modeling relationships between visual features and clinical concepts, we create interpretable concept vectors that guide a multi-agent RAG system to generate radiology reports, enhancing clinical relevance, explainability, and transparency. Evaluation of the generated reports using an LLM-as-a-judge confirmed the interpretability and clinical utility of our model's outputs. On the COVID-QU dataset, our model achieved 81% classification accuracy and demonstrated robust report generation performance, with five key metrics ranging between 84% and 90%. This interpretable multi-agent framework bridges the gap between high-performance AI and the explainability required for reliable AI-driven CXR analysis in clinical settings. Our code is available at https://github.com/tifat58/IRR-with-CBM-RAG.git.