A Simple Framework to Leverage State-Of-The-Art Single-Image Super-Resolution Methods to Restore Light Fields

TL;DR

This paper introduces a straightforward framework that leverages advanced single-image super-resolution techniques to enhance light field images by exploiting their geometrical structure, resulting in improved super-resolution performance with lower complexity.

Contribution

The paper presents a novel framework that combines light field alignment, SVD decomposition, and SISR methods to effectively super-resolve light fields by focusing on the coherent principal component.

Findings

Outperforms recent light field super-resolution methods in PSNR and SSIM.

Achieves super-resolution with lower computational complexity.

Effectively captures coherent information across views using SVD and SISR.

Abstract

Plenoptic cameras offer a cost effective solution to capture light fields by multiplexing multiple views on a single image sensor. However, the high angular resolution is achieved at the expense of reducing the spatial resolution of each view by orders of magnitude compared to the raw sensor image. While light field super-resolution is still at an early stage, the field of single image super-resolution (SISR) has recently known significant advances with the use of deep learning techniques. This paper describes a simple framework allowing us to leverage state-of-the-art SISR techniques into light fields, while taking into account specific light field geometrical constraints. The idea is to first compute a representation compacting most of the light field energy into as few components as possible. This is achieved by aligning the light field using optical flows and then by decomposing the aligned light field using singular value decomposition (SVD). The principal basis captures the information that is coherent across all the views, while the other basis contain the high angular frequencies. Super-resolving this principal basis using an SISR method allows us to super-resolve all the information that is coherent across the entire light field. This framework allows the proposed light field super-resolution method to inherit the benefits of the SISR method used. Experimental results show that the proposed method is competitive, and most of the time superior, to recent light field super-resolution methods in terms of both PSNR and SSIM quality metrics, with a lower complexity.