Epipolar Focus Spectrum: A Novel Light Field Representation and Application in Dense-view Reconstruction

TL;DR

This paper introduces the Epipolar Focus Spectrum (EFS), a new light field representation that is depth-invariant and improves dense-view reconstruction, especially under occlusion and large disparity conditions.

Contribution

The paper proposes the EFS representation, which rearranges EPI spectra for better structural understanding and develops a trainable pipeline for dense light field reconstruction from sparse data.

Findings

EFS is invariant to scene depth and captures view relationships more effectively.

The proposed method outperforms state-of-the-art techniques in reconstruction quality.

Experimental results validate the approach on synthetic and real-world datasets.

Abstract

Existing light field representations, such as epipolar plane image (EPI) and sub-aperture images, do not consider the structural characteristics across the views, so they usually require additional disparity and spatial structure cues for follow-up tasks. Besides, they have difficulties dealing with occlusions or larger disparity scenes. To this end, this paper proposes a novel Epipolar Focus Spectrum (EFS) representation by rearranging the EPI spectrum. Different from the classical EPI representation where an EPI line corresponds to a specific depth, there is a one-to-one mapping from the EFS line to the view. Accordingly, compared to a sparsely-sampled light field, a densely-sampled one with the same field of view (FoV) leads to a more compact distribution of such linear structures in the double-cone-shaped region with the identical opening angle in its corresponding EFS. Hence the EFS representation is invariant to the scene depth. To demonstrate its effectiveness, we develop a trainable EFS-based pipeline for light field reconstruction, where a dense light field can be reconstructed by compensating the "missing EFS lines" given a sparse light field, yielding promising results with cross-view consistency, especially in the presence of severe occlusion and large disparity. Experimental results on both synthetic and real-world datasets demonstrate the validity and superiority of the proposed method over SOTA methods.