Large Language Model-Augmented Auto-Delineation of Treatment Target Volume in Radiation Therapy

TL;DR

This paper introduces Radformer, a novel AI model that combines vision transformers and large language models to improve the accuracy and consistency of delineating radiation therapy targets in cancer treatment, reducing manual effort.

Contribution

The study presents Radformer, a hierarchical vision transformer integrated with large language models and a visual language attention module for enhanced RT target delineation.

Findings

Radformer outperforms existing models in segmentation accuracy.

The model achieves higher DSC, IOU, and lower HD95 scores.

Validated on a large dataset of 2985 patients with head-and-neck cancer.

Abstract

Radiation therapy (RT) is one of the most effective treatments for cancer, and its success relies on the accurate delineation of targets. However, target delineation is a comprehensive medical decision that currently relies purely on manual processes by human experts. Manual delineation is time-consuming, laborious, and subject to interobserver variations. Although the advancements in artificial intelligence (AI) techniques have significantly enhanced the auto-contouring of normal tissues, accurate delineation of RT target volumes remains a challenge. In this study, we propose a visual language model-based RT target volume auto-delineation network termed Radformer. The Radformer utilizes a hierarichal vision transformer as the backbone and incorporates large language models to extract text-rich features from clinical data. We introduce a visual language attention module (VLAM) for integrating visual and linguistic features for language-aware visual encoding (LAVE). The Radformer has been evaluated on a dataset comprising 2985 patients with head-and-neck cancer who underwent RT. Metrics, including the Dice similarity coefficient (DSC), intersection over union (IOU), and 95th percentile Hausdorff distance (HD95), were used to evaluate the performance of the model quantitatively. Our results demonstrate that the Radformer has superior segmentation performance compared to other state-of-the-art models, validating its potential for adoption in RT practice.