Learning Light Field Reconstruction from a Single Coded Image

TL;DR

This paper introduces a deep learning framework that reconstructs a full light field from a single coded image by sequentially estimating the center view, disparity map, and warping to generate the light field, improving over previous methods.

Contribution

It presents a novel three-stage neural network approach for light field reconstruction from a single image, including unsupervised disparity estimation, advancing the state-of-the-art.

Findings

Better parallax recovery from coded images

Outperforms dictionary learning methods quantitatively

Effective unsupervised disparity estimation

Abstract

Light field imaging is a rich way of representing the 3D world around us. However, due to limited sensor resolution capturing light field data inherently poses spatio-angular resolution trade-off. In this paper, we propose a deep learning based solution to tackle the resolution trade-off. Specifically, we reconstruct full sensor resolution light field from a single coded image. We propose to do this in three stages 1) reconstruction of center view from the coded image 2) estimating disparity map from the coded image and center view 3) warping center view using the disparity to generate light field. We propose three neural networks for these stages. Our disparity estimation network is trained in an unsupervised manner alleviating the need for ground truth disparity. Our results demonstrate better recovery of parallax from the coded image. Also, we get better results than dictionary learning based approaches both qualitatively and quatitatively.