Weakly Supervised Fine-grained Span-Level Framework for Chinese Radiology Report Quality Assurance

TL;DR

This paper introduces Sqator, a span-level framework that automatically assesses the quality of junior radiology reports by analyzing fine-grained semantic differences, reducing reliance on labor-intensive senior review.

Contribution

The novel span-level approach for radiology report QA improves accuracy and interpretability over traditional document-level methods, enabling automatic quality evaluation.

Findings

Achieves competitive QA scores on 12,013 reports

Span importance scores align with senior doctors' judgments

Reduces labor costs in radiology report quality assurance

Abstract

Quality Assurance (QA) for radiology reports refers to judging whether the junior reports (written by junior doctors) are qualified. The QA scores of one junior report are given by the senior doctor(s) after reviewing the image and junior report. This process requires intensive labor costs for senior doctors. Additionally, the QA scores may be inaccurate for reasons like diagnosis bias, the ability of senior doctors, and so on. To address this issue, we propose a Span-level Quality Assurance EvaluaTOR (Sqator) to mark QA scores automatically. Unlike the common document-level semantic comparison method, we try to analyze the semantic difference by exploring more fine-grained text spans. Specifically, Sqator measures QA scores by measuring the importance of revised spans between junior and senior reports, and outputs the final QA scores by merging all revised span scores. We evaluate Sqator using a collection of 12,013 radiology reports. Experimental results show that Sqator can achieve competitive QA scores. Moreover, the importance scores of revised spans can be also consistent with the judgments of senior doctors.