EndoWave: Rational-Wavelet 4D Gaussian Splatting for Endoscopic Reconstruction

TL;DR

EndoWave introduces a novel 4D Gaussian splatting framework with optical flow and wavelet constraints to improve 3D endoscopic reconstruction amidst challenging visual artifacts.

Contribution

The paper presents a unified 4D Gaussian splatting method incorporating optical flow and wavelet supervision for more accurate endoscopic 3D reconstruction.

Findings

Achieves state-of-the-art reconstruction quality on real datasets.

Enhances temporal coherence and scene structure accuracy.

Improves rendering performance with wavelet constraints.

Abstract

In robot-assisted minimally invasive surgery, accurate 3D reconstruction from endoscopic video is vital for downstream tasks and improved outcomes. However, endoscopic scenarios present unique challenges, including photometric inconsistencies, non-rigid tissue motion, and view-dependent highlights. Most 3DGS-based methods that rely solely on appearance constraints for optimizing 3DGS are often insufficient in this context, as these dynamic visual artifacts can mislead the optimization process and lead to inaccurate reconstructions. To address these limitations, we present EndoWave, a unified spatio-temporal Gaussian Splatting framework by incorporating an optical flow-based geometric constraint and a multi-resolution rational wavelet supervision. First, we adopt a unified spatio-temporal Gaussian representation that directly optimizes primitives in a 4D domain. Second, we propose a geometric constraint derived from optical flow to enhance temporal coherence and effectively constrain the 3D structure of the scene. Third, we propose a multi-resolution rational orthogonal wavelet as a constraint, which can effectively separate the details of the endoscope and enhance the rendering performance. Extensive evaluations on two real surgical datasets, EndoNeRF and StereoMIS, demonstrate that our method EndoWave achieves state-of-the-art reconstruction quality and visual accuracy compared to the baseline method.