Progressively-connected Light Field Network for Efficient View Synthesis

TL;DR

ProLiF introduces a progressive training scheme for neural light fields that enhances multi-view consistency and rendering quality in view synthesis, outperforming vanilla methods and matching NeRF-like approaches.

Contribution

The paper proposes a novel progressive training scheme and regularization losses for neural light fields, improving multi-view consistency and rendering quality in complex scenes.

Findings

ProLiF achieves better rendering quality than vanilla neural light fields.

ProLiF performs comparably to NeRF-like methods on challenging datasets.

Incorporating LPIPS and CLIP losses enhances robustness and style control.

Abstract

This paper presents a Progressively-connected Light Field network (ProLiF), for the novel view synthesis of complex forward-facing scenes. ProLiF encodes a 4D light field, which allows rendering a large batch of rays in one training step for image- or patch-level losses. Directly learning a neural light field from images has difficulty in rendering multi-view consistent images due to its unawareness of the underlying 3D geometry. To address this problem, we propose a progressive training scheme and regularization losses to infer the underlying geometry during training, both of which enforce the multi-view consistency and thus greatly improves the rendering quality. Experiments demonstrate that our method is able to achieve significantly better rendering quality than the vanilla neural light fields and comparable results to NeRF-like rendering methods on the challenging LLFF dataset and Shiny Object dataset. Moreover, we demonstrate better compatibility with LPIPS loss to achieve robustness to varying light conditions and CLIP loss to control the rendering style of the scene. Project page: https://totoro97.github.io/projects/prolif.