Learning Image Representations for Content Based Image Retrieval of Radiotherapy Treatment Plans

TL;DR

This paper introduces a content-based image retrieval system using deep learning to find similar radiotherapy treatment plans based on anatomical features, aiming to improve upon traditional knowledge-based planning methods.

Contribution

The study develops a novel CBIR approach utilizing Siamese networks for retrieving dose distributions, addressing data limitations in end-to-end deep learning models for radiotherapy planning.

Findings

Siamese networks, especially multitask variants, outperform other encoding methods.

The proposed CBIR method achieves high retrieval accuracy on clinical and public datasets.

Applying CBIR can potentially enhance automated treatment planning workflows.

Abstract

Objective: Knowledge based planning (KBP) typically involves training an end-to-end deep learning model to predict dose distributions. However, training end-to-end methods may be associated with practical limitations due to the limited size of medical datasets that are often used. To address these limitations, we propose a content based image retrieval (CBIR) method for retrieving dose distributions of previously planned patients based on anatomical similarity. Approach: Our proposed CBIR method trains a representation model that produces latent space embeddings of a patient's anatomical information. The latent space embeddings of new patients are then compared against those of previous patients in a database for image retrieval of dose distributions. All source code for this project is available on github. Main Results: The retrieval performance of various CBIR methods is evaluated on a dataset consisting of both publicly available plans and clinical plans from our institution. This study compares various encoding methods, ranging from simple autoencoders to more recent Siamese networks like SimSiam, and the best performance was observed for the multitask Siamese network. Significance: Applying CBIR to inform subsequent treatment planning potentially addresses many limitations associated with end-to-end KBP. Our current results demonstrate that excellent image retrieval performance can be obtained through slight changes to previously developed Siamese networks. We hope to integrate CBIR into automated planning workflow in future works, potentially through methods like the MetaPlanner framework.