Automated Knee X-ray Report Generation

TL;DR

This paper presents a framework that automatically generates diagnostic knee X-ray reports by learning from past exams, effectively translating images into coherent, radiologist-like reports without requiring extensive manual annotations.

Contribution

It introduces a novel method that leverages past radiological exams to train a model capable of generating diagnostic reports from knee X-ray images, reducing the need for manual annotations.

Findings

Generated reports closely match radiologist reports

The method effectively handles multiple image views per exam

Automates report generation to assist clinical workflows

Abstract

Gathering manually annotated images for the purpose of training a predictive model is far more challenging in the medical domain than for natural images as it requires the expertise of qualified radiologists. We therefore propose to take advantage of past radiological exams (specifically, knee X-ray examinations) and formulate a framework capable of learning the correspondence between the images and reports, and hence be capable of generating diagnostic reports for a given X-ray examination consisting of an arbitrary number of image views. We demonstrate how aggregating the image features of individual exams and using them as conditional inputs when training a language generation model results in auto-generated exam reports that correlate well with radiologist-generated reports.