Actor Critic with Experience Replay-based automatic treatment planning for prostate cancer intensity modulated radiotherapy

TL;DR

This paper introduces a DRL-based agent using Actor-Critic with Experience Replay for automatic prostate cancer IMRT treatment planning, achieving high-quality plans with limited training data and robustness against adversarial attacks.

Contribution

It presents a novel DRL approach that generalizes well from a single training case to multiple datasets, improving automation and robustness in treatment planning.

Findings

High plan quality with 93.09% perfect scores

Effective generalization from a single training case

Robustness against adversarial attacks demonstrated

Abstract

Background: Real-time treatment planning in IMRT is challenging due to complex beam interactions. AI has improved automation, but existing models require large, high-quality datasets and lack universal applicability. Deep reinforcement learning (DRL) offers a promising alternative by mimicking human trial-and-error planning. Purpose: Develop a stochastic policy-based DRL agent for automatic treatment planning with efficient training, broad applicability, and robustness against adversarial attacks using Fast Gradient Sign Method (FGSM). Methods: Using the Actor-Critic with Experience Replay (ACER) architecture, the agent tunes treatment planning parameters (TPPs) in inverse planning. Training is based on prostate cancer IMRT cases, using dose-volume histograms (DVHs) as input. The model is trained on a single patient case, validated on two independent cases, and tested on 300+ plans across three datasets. Plan quality is assessed using ProKnow scores, and robustness is tested against adversarial attacks. Results: Despite training on a single case, the model generalizes well. Before ACER-based planning, the mean plan score was 6.20$\pm$1.84; after, 93.09% of cases achieved a perfect score of 9, with a mean of 8.93$\pm$0.27. The agent effectively prioritizes optimal TPP tuning and remains robust against adversarial attacks. Conclusions: The ACER-based DRL agent enables efficient, high-quality treatment planning in prostate cancer IMRT, demonstrating strong generalizability and robustness.