Using Large Language Models To Translate Machine Results To Human Results

TL;DR

This paper presents a pipeline combining object detection models and large language models to automatically generate human-like radiology reports from chest X-ray findings, aiming to improve interpretability and clinical utility.

Contribution

It introduces an integrated system that translates structured AI detection outputs into natural language reports using LLMs, demonstrating improved semantic similarity and report quality.

Findings

Strong semantic similarity between AI-generated and ground-truth reports

GPT-4 achieves high clarity scores in report generation

Current systems produce reports stylistically distinguishable from radiologists' writing

Abstract

Artificial intelligence (AI) has transformed medical imaging, with computer vision (CV) systems achieving state-of-the-art performance in classification and detection tasks. However, these systems typically output structured predictions, leaving radiologists responsible for translating results into full narrative reports. Recent advances in large language models (LLMs), such as GPT-4, offer new opportunities to bridge this gap by generating diagnostic narratives from structured findings. This study introduces a pipeline that integrates YOLOv5 and YOLOv8 for anomaly detection in chest X-ray images with a large language model (LLM) to generate natural-language radiology reports. The YOLO models produce bounding-box predictions and class labels, which are then passed to the LLM to generate descriptive findings and clinical summaries. YOLOv5 and YOLOv8 are compared in terms of detection accuracy, inference latency, and the quality of generated text, as measured by cosine similarity to ground-truth reports. Results show strong semantic similarity between AI and human reports, while human evaluation reveals GPT-4 excels in clarity (4.88/5) but exhibits lower scores for natural writing flow (2.81/5), indicating that current systems achieve clinical accuracy but remain stylistically distinguishable from radiologist-authored text.