Warm-Started Reinforcement Learning for Iterative 3D/2D Liver Registration

TL;DR

This paper introduces a reinforcement learning framework for efficient, automated 3D/2D liver registration in surgical AR, achieving comparable accuracy to traditional methods with faster convergence.

Contribution

It presents a warm-started RL approach that formulates registration as a sequential decision process, improving speed and automation over existing supervised methods.

Findings

Achieved an average TRE of 15.70 mm on a public dataset.

Faster convergence compared to supervised optimization-based methods.

Automated registration without manual tuning of step sizes or stopping criteria.

Abstract

Registration between preoperative CT and intraoperative laparoscopic video plays a crucial role in augmented reality (AR) guidance for minimally invasive surgery. Learning-based methods have recently achieved registration errors comparable to optimization-based approaches while offering faster inference. However, many supervised methods produce coarse alignments that rely on additional optimization-based refinement, thereby increasing inference time. We present a discrete-action reinforcement learning (RL) framework that formulates CT-to-video registration as a sequential decision-making process. A shared feature encoder, warm-started from a supervised pose estimation network to provide stable geometric features and faster convergence, extracts representations from CT renderings and laparoscopic frames, while an RL policy head learns to choose rigid transformations along six degrees of freedom and to decide when to stop the iteration. Experiments on a public laparoscopic dataset demonstrated that our method achieved an average target registration error (TRE) of 15.70 mm, comparable to supervised approaches with optimization, while achieving faster convergence. The proposed RL-based formulation enables automated, efficient iterative registration without manually tuned step sizes or stopping criteria. This discrete framework provides a practical foundation for future continuous-action and deformable registration models in surgical AR applications.