Plenodium: UnderWater 3D Scene Reconstruction with Plenoptic Medium Representation

TL;DR

Plenodium introduces a novel plenoptic medium representation for underwater 3D scene reconstruction, combining directional and positional data to improve accuracy in degraded environments.

Contribution

The paper proposes a new plenoptic medium model with spherical harmonics encoding and robust depth priors, enhancing underwater scene reconstruction accuracy.

Findings

Significant improvement in 3D reconstruction quality on real-world underwater datasets.

Effective scene restoration demonstrated in simulated environments with controllable scattering.

Robust depth estimation achieved through pseudo-depth Gaussian complementation and depth ranking regularization.

Abstract

We present Plenodium (plenoptic medium), an effective and efficient 3D representation framework capable of jointly modeling both objects and participating media. In contrast to existing medium representations that rely solely on view-dependent modeling, our novel plenoptic medium representation incorporates both directional and positional information through spherical harmonics encoding, enabling highly accurate underwater scene reconstruction. To address the initialization challenge in degraded underwater environments, we propose the pseudo-depth Gaussian complementation to augment COLMAP-derived point clouds with robust depth priors. In addition, a depth ranking regularized loss is developed to optimize the geometry of the scene and improve the ordinal consistency of the depth maps. Extensive experiments on real-world underwater datasets demonstrate that our method achieves significant improvements in 3D reconstruction. Furthermore, we conduct a simulated dataset with ground truth and the controllable scattering medium to demonstrate the restoration capability of our method in underwater scenarios. Our code and dataset are available at https://plenodium.github.io/.