Deep Spatial-angular Regularization for Compressive Light Field Reconstruction over Coded Apertures

TL;DR

This paper introduces a deep learning framework that improves the quality of light field reconstruction from coded aperture measurements by integrating measurement data into the learning process, resulting in higher fidelity and robustness.

Contribution

The proposed method uniquely combines measurement observation with deep spatial-angular regularization, enhancing reconstruction quality over existing algorithms.

Findings

Achieves higher PSNR and SSIM compared to state-of-the-art methods.

Better preserves light field parallax structure.

Demonstrates robustness to noise and efficiency in reconstruction.

Abstract

Coded aperture is a promising approach for capturing the 4-D light field (LF), in which the 4-D data are compressively modulated into 2-D coded measurements that are further decoded by reconstruction algorithms. The bottleneck lies in the reconstruction algorithms, resulting in rather limited reconstruction quality. To tackle this challenge, we propose a novel learning-based framework for the reconstruction of high-quality LFs from acquisitions via learned coded apertures. The proposed method incorporates the measurement observation into the deep learning framework elegantly to avoid relying entirely on data-driven priors for LF reconstruction. Specifically, we first formulate the compressive LF reconstruction as an inverse problem with an implicit regularization term. Then, we construct the regularization term with an efficient deep spatial-angular convolutional sub-network to comprehensively explore the signal distribution free from the limited representation ability and inefficiency of deterministic mathematical modeling. Experimental results show that the reconstructed LFs not only achieve much higher PSNR/SSIM but also preserve the LF parallax structure better, compared with state-of-the-art methods on both real and synthetic LF benchmarks. In addition, experiments show that our method is efficient and robust to noise, which is an essential advantage for a real camera system. The code is publicly available at \url{https://github.com/angmt2008/LFCA}