Surgical Neural Radiance Fields from One Image

TL;DR

This paper introduces a method to train Neural Radiance Fields (NeRF) using only a single intraoperative image combined with preoperative data, enabling fast and effective 3D reconstruction in surgical scenarios with limited data.

Contribution

The novel approach leverages preoperative MRI and neural style transfer to enable single-image NeRF training for intraoperative use, overcoming multi-view data limitations.

Findings

High reconstruction fidelity in neurosurgical cases

Effective style transfer preserves surgical image textures

Fast training suitable for intraoperative applications

Abstract

Purpose: Neural Radiance Fields (NeRF) offer exceptional capabilities for 3D reconstruction and view synthesis, yet their reliance on extensive multi-view data limits their application in surgical intraoperative settings where only limited data is available. In particular, collecting such extensive data intraoperatively is impractical due to time constraints. This work addresses this challenge by leveraging a single intraoperative image and preoperative data to train NeRF efficiently for surgical scenarios. Methods: We leverage preoperative MRI data to define the set of camera viewpoints and images needed for robust and unobstructed training. Intraoperatively, the appearance of the surgical image is transferred to the pre-constructed training set through neural style transfer, specifically combining WTC2 and STROTSS to prevent over-stylization. This process enables the creation of a dataset for instant and fast single-image NeRF training. Results: The method is evaluated with four clinical neurosurgical cases. Quantitative comparisons to NeRF models trained on real surgical microscope images demonstrate strong synthesis agreement, with similarity metrics indicating high reconstruction fidelity and stylistic alignment. When compared with ground truth, our method demonstrates high structural similarity, confirming good reconstruction quality and texture preservation. Conclusion: Our approach demonstrates the feasibility of single-image NeRF training in surgical settings, overcoming the limitations of traditional multi-view methods.