EndoSurf: Neural Surface Reconstruction of Deformable Tissues with Stereo Endoscope Videos

TL;DR

EndoSurf introduces a neural-field-based approach for high-fidelity reconstruction of deformable tissues from stereo endoscope videos, effectively modeling shape, deformation, and appearance to improve medical imaging applications.

Contribution

The paper presents a novel neural surface reconstruction method that models deforming tissues using three neural fields, outperforming existing solutions in accuracy and detail.

Findings

Significantly outperforms existing methods in shape reconstruction quality.

Effectively models tissue deformation, shape, and appearance.

Achieves high-fidelity reconstructions on public datasets.

Abstract

Reconstructing soft tissues from stereo endoscope videos is an essential prerequisite for many medical applications. Previous methods struggle to produce high-quality geometry and appearance due to their inadequate representations of 3D scenes. To address this issue, we propose a novel neural-field-based method, called EndoSurf, which effectively learns to represent a deforming surface from an RGBD sequence. In EndoSurf, we model surface dynamics, shape, and texture with three neural fields. First, 3D points are transformed from the observed space to the canonical space using the deformation field. The signed distance function (SDF) field and radiance field then predict their SDFs and colors, respectively, with which RGBD images can be synthesized via differentiable volume rendering. We constrain the learned shape by tailoring multiple regularization strategies and disentangling geometry and appearance. Experiments on public endoscope datasets demonstrate that EndoSurf significantly outperforms existing solutions, particularly in reconstructing high-fidelity shapes. Code is available at https://github.com/Ruyi-Zha/endosurf.git.