Multi-View Reconstruction using Signed Ray Distance Functions (SRDF)

TL;DR

This paper introduces a novel volumetric shape representation called Signed Ray Distance Functions (SRDF) that combines the pixel accuracy of multi-view stereo with the volumetric consistency of differentiable rendering, improving 3D reconstruction precision.

Contribution

The paper proposes a new implicit volumetric shape representation parameterized by pixel depths, bridging multi-view stereo and differentiable rendering for enhanced 3D reconstruction accuracy.

Findings

Outperforms existing methods on standard 3D benchmarks.

Achieves better geometric estimations with volumetric integration.

Maintains pixel-accuracy while enabling volumetric optimization.

Abstract

In this paper, we investigate a new optimization framework for multi-view 3D shape reconstructions. Recent differentiable rendering approaches have provided breakthrough performances with implicit shape representations though they can still lack precision in the estimated geometries. On the other hand multi-view stereo methods can yield pixel wise geometric accuracy with local depth predictions along viewing rays. Our approach bridges the gap between the two strategies with a novel volumetric shape representation that is implicit but parameterized with pixel depths to better materialize the shape surface with consistent signed distances along viewing rays. The approach retains pixel-accuracy while benefiting from volumetric integration in the optimization. To this aim, depths are optimized by evaluating, at each 3D location within the volumetric discretization, the agreement between the depth prediction consistency and the photometric consistency for the corresponding pixels. The optimization is agnostic to the associated photo-consistency term which can vary from a median-based baseline to more elaborate criteria learned functions. Our experiments demonstrate the benefit of the volumetric integration with depth predictions. They also show that our approach outperforms existing approaches over standard 3D benchmarks with better geometry estimations.