Pl\"uckeRF: A Line-based 3D Representation for Few-view Reconstruction

TL;DR

PlückeRF introduces a line-based 3D representation that enhances few-view scene reconstruction by effectively integrating multi-view information, leading to improved accuracy over existing methods.

Contribution

The paper presents a novel structured line-based 3D representation that better leverages multi-view data for feed-forward reconstruction.

Findings

Improved reconstruction quality over triplane representations.

Enhanced performance compared to state-of-the-art feedforward methods.

Effective multi-view information sharing through structured lines.

Abstract

Feed-forward 3D reconstruction methods aim to predict the 3D structure of a scene directly from input images, providing a faster alternative to per-scene optimization approaches. Significant progress has been made in single-view and few-view reconstruction using learned priors that infer object shape and appearance, even for unobserved regions. However, there is substantial potential to enhance these methods by better leveraging information from multiple views when available. To address this, we propose a few-view reconstruction model that more effectively harnesses multi-view information. Our approach introduces a simple mechanism that connects the 3D representation with pixel rays from the input views, allowing for preferential sharing of information between nearby 3D locations and between 3D locations and nearby pixel rays. We achieve this by defining the 3D representation as a set of structured, feature-augmented lines; the Pl\"uckeRF representation. Using this representation, we demonstrate improvements in reconstruction quality over the equivalent triplane representation and state-of-the-art feedforward reconstruction methods.