View Correspondence Network for Implicit Light Field Representation

TL;DR

This paper introduces VICON, a novel implicit neural network for light field representation that achieves high fidelity, multi-view consistency, and larger field of view by leveraging stereo matching and pixel correspondence.

Contribution

VICON uniquely integrates stereo matching and multi-view correspondence into an implicit neural light field model, improving view consistency and extending the observable scene.

Findings

VICON outperforms existing non-3D implicit light field methods both qualitatively and quantitatively.

VICON captures a larger field of view than the original camera setup.

The method maintains high quality and fidelity with low storage and inference costs.

Abstract

We present a novel technique for implicit neural representation of light fields at continuously defined viewpoints with high quality and fidelity. Our implicit neural representation maps 4D coordinates defining two-plane parameterization of the light fields to the corresponding color values. We leverage periodic activations to achieve high expressivity and accurate reconstruction for complex data manifolds while keeping low storage and inference time requirements. However, na\"ively trained non-3D structured networks do not adequately satisfy the multi-view consistency; instead, they perform alpha blending of nearby viewpoints. In contrast, our View Correspondence Network, or VICON, leverages stereo matching, optimization by automatic differentiation with respect to the input space, and multi-view pixel correspondence to provide a novel implicit representation of the light fields faithful to the novel views that are unseen during the training. Experimental results show VICON superior to the state-of-the-art non-3D implicit light field representations both qualitatively and quantitatively. Moreover, our implicit representation captures a larger field of view (FoV), surpassing the extent of the observable scene by the cameras of the ground truth renderings.