Learning Sub-Pixel Disparity Distribution for Light Field Depth Estimation

TL;DR

This paper introduces a novel approach for light field depth estimation that models the full disparity distribution at sub-pixel resolution, leading to significantly improved accuracy over existing methods.

Contribution

It proposes a method to learn the sub-pixel disparity distribution using a cost volume and an uncertainty-aware focal loss, enhancing depth estimation precision.

Findings

Outperforms state-of-the-art LF depth algorithms on HCI 4D LF Benchmark

Achieves better accuracy metrics including BadPix and MSE

Effectively models disparity distribution at sub-pixel level

Abstract

Light field (LF) depth estimation plays a crucial role in many LF-based applications. Existing LF depth estimation methods consider depth estimation as a regression problem, where a pixel-wise L1 loss is employed to supervise the training process. However, the disparity map is only a sub-space projection (i.e., an expectation) of the disparity distribution, which is essential for models to learn. In this paper, we propose a simple yet effective method to learn the sub-pixel disparity distribution by fully utilizing the power of deep networks, especially for LF of narrow baselines. We construct the cost volume at the sub-pixel level to produce a finer disparity distribution and design an uncertainty-aware focal loss to supervise the predicted disparity distribution toward the ground truth. Extensive experimental results demonstrate the effectiveness of our method.Our method significantly outperforms recent state-of-the-art LF depth algorithms on the HCI 4D LF Benchmark in terms of all four accuracy metrics (i.e., BadPix 0.01, BadPix 0.03, BadPix 0.07, and MSE $\times$100). The code and model of the proposed method are available at \url{https://github.com/chaowentao/SubFocal}.