DeepView: View Synthesis with Learned Gradient Descent

TL;DR

DeepView introduces a learned gradient descent approach to generate multiplane images for view synthesis, effectively handling occlusions and complex scene features to produce high-quality, state-of-the-art results.

Contribution

It presents a novel learned gradient descent method for MPI generation from sparse viewpoints, enhancing occlusion reasoning and scene detail handling.

Findings

Achieves state-of-the-art results on Kalantari light field dataset

Performs well on the new Spaces dataset with complex scenes

Improves rendering quality for challenging scene features

Abstract

We present a novel approach to view synthesis using multiplane images (MPIs). Building on recent advances in learned gradient descent, our algorithm generates an MPI from a set of sparse camera viewpoints. The resulting method incorporates occlusion reasoning, improving performance on challenging scene features such as object boundaries, lighting reflections, thin structures, and scenes with high depth complexity. We show that our method achieves high-quality, state-of-the-art results on two datasets: the Kalantari light field dataset, and a new camera array dataset, Spaces, which we make publicly available.