Extreme View Synthesis

TL;DR

Extreme View Synthesis introduces a novel approach for extrapolating new views from as few as two images, effectively handling occlusions and depth uncertainty to produce high-quality results even at large magnifications.

Contribution

The paper proposes estimating a depth probability volume instead of a single depth, and combines learned priors with depth uncertainty for improved view synthesis.

Findings

Achieves visually pleasing results up to 30X magnification.

Effectively handles occlusions and depth uncertainty in extrapolation.

Outperforms previous methods in small input image scenarios.

Abstract

We present Extreme View Synthesis, a solution for novel view extrapolation that works even when the number of input images is small--as few as two. In this context, occlusions and depth uncertainty are two of the most pressing issues, and worsen as the degree of extrapolation increases. We follow the traditional paradigm of performing depth-based warping and refinement, with a few key improvements. First, we estimate a depth probability volume, rather than just a single depth value for each pixel of the novel view. This allows us to leverage depth uncertainty in challenging regions, such as depth discontinuities. After using it to get an initial estimate of the novel view, we explicitly combine learned image priors and the depth uncertainty to synthesize a refined image with less artifacts. Our method is the first to show visually pleasing results for baseline magnifications of up to 30X.