RAFM: Retrieval-Augmented Flow Matching for Unpaired CBCT-to-CT Translation

TL;DR

This paper introduces Retrieval-Augmented Flow Matching (RAFM), a novel method for unpaired CBCT-to-CT translation that leverages retrieval-guided pseudo pairs to improve stability and performance in medical imaging tasks.

Contribution

The paper proposes RAFM, which adapts flow matching with retrieval guidance for unpaired medical image translation, addressing stability issues with small datasets.

Findings

RAFM outperforms existing methods on SynthRAD2023 dataset.

Retrieval-guided pseudo pairs improve training stability.

Method achieves better FID, MAE, SSIM, PSNR, and SegScore metrics.

Abstract

Cone-beam CT (CBCT) is routinely acquired in radiotherapy but suffers from severe artifacts and unreliable Hounsfield Unit (HU) values, limiting its direct use for dose calculation. Synthetic CT (sCT) generation from CBCT is therefore an important task, yet paired CBCT--CT data are often unavailable or unreliable due to temporal gaps, anatomical variation, and registration errors. In this work, we introduce rectified flow (RF) into unpaired CBCT-to-CT translation in medical imaging. Although RF is theoretically compatible with unpaired learning through distribution-level coupling and deterministic transport, its practical effectiveness under small medical datasets and limited batch sizes remains underexplored. Direct application with random or batch-local pseudo pairing can produce unstable supervision due to semantically mismatched endpoint samples. To address this challenge, we propose Retrieval-Augmented Flow Matching (RAFM), which adapts RF to the medical setting by constructing retrieval-guided pseudo pairs using a frozen DINOv3 encoder and a global CT memory bank. This strategy improves empirical coupling quality and stabilizes unpaired flow-based training. Experiments on SynthRAD2023 under a strict subject-level true-unpaired protocol show that RAFM outperforms existing methods across FID, MAE, SSIM, PSNR, and SegScore. The code is available at https://github.com/HiLab-git/RAFM.git.