MvKeTR: Chest CT Report Generation with Multi-View Perception and Knowledge Enhancement

TL;DR

This paper introduces MvKeTR, a novel transformer-based framework that mimics radiologists' multi-view analysis and knowledge retrieval to improve automatic chest CT report generation, outperforming existing models.

Contribution

The paper proposes a multi-view perception and knowledge enhancement approach using Kolmogorov-Arnold Networks, advancing CT report generation with better integration of diagnostic views and clinical knowledge.

Findings

Outperforms state-of-the-art models on CTRG-Chest-548 dataset

Effectively integrates multi-view diagnostic information

Utilizes KANs for improved parameter efficiency and high-frequency feature capture

Abstract

CT report generation (CTRG) aims to automatically generate diagnostic reports for 3D volumes, relieving clinicians' workload and improving patient care. Despite clinical value, existing works fail to effectively incorporate diagnostic information from multiple anatomical views and lack related clinical expertise essential for accurate and reliable diagnosis. To resolve these limitations, we propose a novel Multi-view perception Knowledge-enhanced TansfoRmer (MvKeTR) to mimic the diagnostic workflow of clinicians. Just as radiologists first examine CT scans from multiple planes, a Multi-View Perception Aggregator (MVPA) with view-aware attention is proposed to synthesize diagnostic information from multiple anatomical views effectively. Then, inspired by how radiologists further refer to relevant clinical records to guide diagnostic decision-making, a Cross-Modal Knowledge Enhancer (CMKE) is devised to retrieve the most similar reports based on the query volume to incorporate domain knowledge into the diagnosis procedure. Furthermore, instead of traditional MLPs, we employ Kolmogorov-Arnold Networks (KANs) as the fundamental building blocks of both modules, which exhibit superior parameter efficiency and reduced spectral bias to better capture high-frequency components critical for CT interpretation while mitigating overfitting. Extensive experiments on the public CTRG-Chest-548 K dataset demonstrate that our method outpaces prior state-of-the-art (SOTA) models across almost all metrics. The code is available at https://github.com/xiweideng/MvKeTR.