Iterative Occlusion-Aware Light Field Depth Estimation using 4D Geometrical Cues

TL;DR

This paper introduces a novel non-learning-based method for light field depth estimation that explicitly exploits 4D geometric cues, improving surface normal accuracy and occlusion handling over existing methods.

Contribution

It presents a fully explainable 4D geometric model for depth estimation that outperforms current state-of-the-art methods in surface normal accuracy.

Findings

Median Angle Error on planar surfaces reduced by 26.3%

Outperforms state-of-the-art in surface normal accuracy

Competitive in MSE and Badpix metrics

Abstract

Light field cameras and multi-camera arrays have emerged as promising solutions for accurately estimating depth by passively capturing light information. This is possible because the 3D information of a scene is embedded in the 4D light field geometry. Commonly, depth estimation methods extract this information relying on gradient information, heuristic-based optimisation models, or learning-based approaches. This paper focuses mainly on explicitly understanding and exploiting 4D geometrical cues for light field depth estimation. Thus, a novel method is proposed, based on a non-learning-based optimisation approach for depth estimation that explicitly considers surface normal accuracy and occlusion regions by utilising a fully explainable 4D geometric model of the light field. The 4D model performs depth/disparity estimation by determining the orientations and analysing the intersections of key 2D planes in 4D space, which are the images of 3D-space points in the 4D light field. Experimental results show that the proposed method outperforms both learning-based and non-learning-based state-of-the-art methods in terms of surface normal angle accuracy, achieving a Median Angle Error on planar surfaces, on average, 26.3$\%$ lower than the state-of-the-art, and still being competitive with state-of-the-art methods in terms of MSE ${\times}$ 100 and Badpix 0.07.