Transforming Multimodal Models into Action Models for Radiotherapy

TL;DR

This paper introduces a novel approach that transforms large multimodal foundation models into action models for radiotherapy treatment planning, improving efficiency and plan quality through few-shot reinforcement learning and simulation.

Contribution

It presents a new framework that leverages pre-trained multimodal models and adapts them for treatment planning using few-shot RL, enhancing accuracy and efficiency.

Findings

Outperforms traditional RL methods in treatment plan quality.

Achieves higher reward scores and better dose distributions in simulations.

Demonstrates potential for clinical workflow integration.

Abstract

Radiotherapy is a crucial cancer treatment that demands precise planning to balance tumor eradication and preservation of healthy tissue. Traditional treatment planning (TP) is iterative, time-consuming, and reliant on human expertise, which can potentially introduce variability and inefficiency. We propose a novel framework to transform a large multimodal foundation model (MLM) into an action model for TP using a few-shot reinforcement learning (RL) approach. Our method leverages the MLM's extensive pre-existing knowledge of physics, radiation, and anatomy, enhancing it through a few-shot learning process. This allows the model to iteratively improve treatment plans using a Monte Carlo simulator. Our results demonstrate that this method outperforms conventional RL-based approaches in both quality and efficiency, achieving higher reward scores and more optimal dose distributions in simulations on prostate cancer data. This proof-of-concept suggests a promising direction for integrating advanced AI models into clinical workflows, potentially enhancing the speed, quality, and standardization of radiotherapy treatment planning.