Fast and Explicit Neural View Synthesis

TL;DR

This paper introduces a simple explicit volumetric approach for neural view synthesis that outperforms or matches state-of-the-art methods in quality while being significantly faster, and it generalizes well across categories without scene-specific training.

Contribution

The authors propose an explicit volumetric encoding method for view synthesis that is faster, category-agnostic, and capable of self-supervised 3D learning, challenging the trend towards implicit representations.

Findings

Achieves over 400x faster rendering than implicit methods.

Performs comparably or better in novel view reconstruction.

Generalizes to unseen object categories without scene-specific optimization.

Abstract

We study the problem of novel view synthesis from sparse source observations of a scene comprised of 3D objects. We propose a simple yet effective approach that is neither continuous nor implicit, challenging recent trends on view synthesis. Our approach explicitly encodes observations into a volumetric representation that enables amortized rendering. We demonstrate that although continuous radiance field representations have gained a lot of attention due to their expressive power, our simple approach obtains comparable or even better novel view reconstruction quality comparing with state-of-the-art baselines while increasing rendering speed by over 400x. Our model is trained in a category-agnostic manner and does not require scene-specific optimization. Therefore, it is able to generalize novel view synthesis to object categories not seen during training. In addition, we show that with our simple formulation, we can use view synthesis as a self-supervision signal for efficient learning of 3D geometry without explicit 3D supervision.