Light Field Super-Resolution using a Low-Rank Prior and Deep Convolutional Neural Networks

TL;DR

This paper introduces a novel learning-based light field super-resolution method that combines optical flow, low-rank approximation, and deep neural networks to enhance spatial resolution while maintaining consistency across sub-aperture images.

Contribution

It proposes a new framework integrating low-rank modeling and deep learning for light field super-resolution, improving over existing methods.

Findings

Outperforms existing algorithms in PSNR by 0.23 dB.

Uses low-rank approximation to reduce angular dimension effectively.

Achieves better spatial resolution with consistency across views.

Abstract

Light field imaging has recently known a regain of interest due to the availability of practical light field capturing systems that offer a wide range of applications in the field of computer vision. However, capturing high-resolution light fields remains technologically challenging since the increase in angular resolution is often accompanied by a significant reduction in spatial resolution. This paper describes a learning-based spatial light field super-resolution method that allows the restoration of the entire light field with consistency across all sub-aperture images. The algorithm first uses optical flow to align the light field and then reduces its angular dimension using low-rank approximation. We then consider the linearly independent columns of the resulting low-rank model as an embedding, which is restored using a deep convolutional neural network (DCNN). The super-resolved embedding is then used to reconstruct the remaining sub-aperture images. The original disparities are restored using inverse warping where missing pixels are approximated using a novel light field inpainting algorithm. Experimental results show that the proposed method outperforms existing light field super-resolution algorithms, achieving PSNR gains of 0.23 dB over the second best performing method. This performance can be further improved using iterative back-projection as a post-processing step.