Depth estimation from 4D light field videos

TL;DR

This paper introduces an end-to-end neural network for depth estimation from 4D light field videos, leveraging temporal information to improve accuracy, and provides a synthetic dataset for training and evaluation.

Contribution

It presents the first neural network architecture specifically designed for depth estimation from 4D LF videos, incorporating temporal data and offering a new synthetic dataset.

Findings

Temporal information improves depth accuracy in noisy regions.

The proposed method outperforms static image-based approaches.

Synthetic and real-world experiments validate effectiveness.

Abstract

Depth (disparity) estimation from 4D Light Field (LF) images has been a research topic for the last couple of years. Most studies have focused on depth estimation from static 4D LF images while not considering temporal information, i.e., LF videos. This paper proposes an end-to-end neural network architecture for depth estimation from 4D LF videos. This study also constructs a medium-scale synthetic 4D LF video dataset that can be used for training deep learning-based methods. Experimental results using synthetic and real-world 4D LF videos show that temporal information contributes to the improvement of depth estimation accuracy in noisy regions. Dataset and code is available at: https://mediaeng-lfv.github.io/LFV_Disparity_Estimation