The Dresden Dataset for 4D Reconstruction of Non-Rigid Abdominal Surgical Scenes

TL;DR

The Dresden Dataset offers a comprehensive collection of endoscopic videos and structured-light geometry data for evaluating 3D reconstruction of deforming abdominal tissues in realistic surgical scenarios, supporting advanced algorithm development.

Contribution

This paper introduces a large, curated dataset combining stereo endoscopic videos and structured-light data for non-rigid 3D reconstruction in surgical environments, enabling robust algorithm benchmarking.

Findings

Dataset includes over 300,000 frames and 369 point clouds.

Supports evaluation of non-rigid SLAM and 4D reconstruction methods.

Provides baseline photometric view-synthesis results.

Abstract

The D4D Dataset provides paired endoscopic video and high-quality structured-light geometry for evaluating 3D reconstruction of deforming abdominal soft tissue in realistic surgical conditions. Data were acquired from six porcine cadaver sessions using a da Vinci Xi stereo endoscope and a Zivid structured-light camera, registered via optical tracking and manually curated iterative alignment methods. Three sequence types - whole deformations, incremental deformations, and moved-camera clips - probe algorithm robustness to non-rigid motion, deformation magnitude, and out-of-view updates. Each clip provides rectified stereo images, per-frame instrument masks, stereo depth, start/end structured-light point clouds, curated camera poses and camera intrinsics. In postprocessing, ICP and semi-automatic registration techniques are used to register data, and instrument masks are created. The dataset enables quantitative geometric evaluation in both visible and occluded regions, alongside photometric view-synthesis baselines. Comprising over 300,000 frames and 369 point clouds across 98 curated recordings, this resource can serve as a comprehensive benchmark for developing and evaluating non-rigid SLAM, 4D reconstruction, and depth estimation methods.