GuidedRec: Guiding Ill-Posed Unsupervised Volumetric Recovery

TL;DR

GuidedRec is an unsupervised method that reconstructs 3D volumes from minimal projections by leveraging prior volume data and generative models, improving accuracy and flexibility in medical imaging.

Contribution

It introduces a generative model-based approach for 3D reconstruction from two projections that does not require sensor calibration retraining.

Findings

Outperforms state-of-the-art methods on challenging datasets.

Effective with very few projections, even two.

Applicable to various sensor calibrations without retraining.

Abstract

We introduce a novel unsupervised approach to reconstructing a 3D volume from only two planar projections that exploits a previous\-ly-captured 3D volume of the patient. Such volume is readily available in many important medical procedures and previous methods already used such a volume. Earlier methods that work by deforming this volume to match the projections typically fail when the number of projections is very low as the alignment becomes underconstrained. We show how to use a generative model of the volume structures to constrain the deformation and obtain a correct estimate. Moreover, our method is not bounded to a specific sensor calibration and can be applied to new calibrations without retraining. We evaluate our approach on a challenging dataset and show it outperforms state-of-the-art methods. As a result, our method could be used in treatment scenarios such as surgery and radiotherapy while drastically reducing patient radiation exposure.