3DVSR: 3D EPI Volume-based Approach for Angular and Spatial Light field Image Super-resolution

TL;DR

This paper introduces a 3D EPI volume-based learning method for high-resolution light field reconstruction, effectively improving spatial and angular super-resolution with a two-stage framework and a novel refinement network.

Contribution

It proposes a novel 2-stage super-resolution framework with an EPI volume-based refinement network for enhanced light field super-resolution.

Findings

Outperforms state-of-the-art methods in spatial and angular super-resolution.

Achieves over 2.0 dB PSNR improvement in spatial SR and 3.14 dB in angular SR.

Demonstrates balanced and superior visual quality across multiple datasets.

Abstract

Light field (LF) imaging, which captures both spatial and angular information of a scene, is undoubtedly beneficial to numerous applications. Although various techniques have been proposed for LF acquisition, achieving both angularly and spatially high-resolution LF remains a technology challenge. In this paper, a learning-based approach applied to 3D epipolar image (EPI) is proposed to reconstruct high-resolution LF. Through a 2-stage super-resolution framework, the proposed approach effectively addresses various LF super-resolution (SR) problems, i.e., spatial SR, angular SR, and angular-spatial SR. While the first stage provides flexible options to up-sample EPI volume to the desired resolution, the second stage, which consists of a novel EPI volume-based refinement network (EVRN), substantially enhances the quality of the high-resolution EPI volume. An extensive evaluation on 90 challenging synthetic and real-world light field scenes from 7 published datasets shows that the proposed approach outperforms state-of-the-art methods to a large extend for both spatial and angular super-resolution problem, i.e., an average peak signal to noise ratio improvement of more than 2.0 dB, 1.4 dB, and 3.14 dB in spatial SR $\times 2$, spatial SR $\times 4$, and angular SR respectively. The reconstructed 4D light field demonstrates a balanced performance distribution across all perspective images and presents superior visual quality compared to the previous works.