360Recon: An Accurate Reconstruction Method Based on Depth Fusion from 360 Images

TL;DR

360Recon is a novel multi-view stereo algorithm that effectively addresses distortion in 360-degree images, enabling accurate and efficient 3D scene reconstruction in low-texture environments for VR and AR applications.

Contribution

The paper introduces a spherical feature extraction module and a combined 3D cost volume approach, advancing the accuracy and efficiency of 360-degree image reconstruction.

Findings

Achieves state-of-the-art performance in depth estimation.

Demonstrates high efficiency in 3D reconstruction.

Maintains local geometric consistency in reconstructed scenes.

Abstract

360-degree images offer a significantly wider field of view compared to traditional pinhole cameras, enabling sparse sampling and dense 3D reconstruction in low-texture environments. This makes them crucial for applications in VR, AR, and related fields. However, the inherent distortion caused by the wide field of view affects feature extraction and matching, leading to geometric consistency issues in subsequent multi-view reconstruction. In this work, we propose 360Recon, an innovative MVS algorithm for ERP images. The proposed spherical feature extraction module effectively mitigates distortion effects, and by combining the constructed 3D cost volume with multi-scale enhanced features from ERP images, our approach achieves high-precision scene reconstruction while preserving local geometric consistency. Experimental results demonstrate that 360Recon achieves state-of-the-art performance and high efficiency in depth estimation and 3D reconstruction on existing public panoramic reconstruction datasets.