Learning Light Field Angular Super-Resolution via a Geometry-Aware Network

TL;DR

This paper presents a geometry-aware neural network for angular super-resolution of light fields with large baselines, explicitly modeling scene geometry and synthesizing novel views to improve quality and preserve parallax.

Contribution

It introduces a novel end-to-end learning framework combining geometry modeling, physically-based warping, and light field blending for large baseline light field super-resolution.

Findings

Outperforms state-of-the-art methods with up to 2 dB PSNR improvement.

Reduces computation time by 48 times.

Better preserves light field parallax structure.

Abstract

The acquisition of light field images with high angular resolution is costly. Although many methods have been proposed to improve the angular resolution of a sparsely-sampled light field, they always focus on the light field with a small baseline, which is captured by a consumer light field camera. By making full use of the intrinsic \textit{geometry} information of light fields, in this paper we propose an end-to-end learning-based approach aiming at angularly super-resolving a sparsely-sampled light field with a large baseline. Our model consists of two learnable modules and a physically-based module. Specifically, it includes a depth estimation module for explicitly modeling the scene geometry, a physically-based warping for novel views synthesis, and a light field blending module specifically designed for light field reconstruction. Moreover, we introduce a novel loss function to promote the preservation of the light field parallax structure. Experimental results over various light field datasets including large baseline light field images demonstrate the significant superiority of our method when compared with state-of-the-art ones, i.e., our method improves the PSNR of the second best method up to 2 dB in average, while saves the execution time 48$\times$. In addition, our method preserves the light field parallax structure better.