Iterative approach to reconstructing neural disparity fields from light-field data

TL;DR

This paper introduces a neural disparity field (NDF) that provides an implicit, continuous representation of scene disparities from light-field data, enabling high-resolution disparity reconstruction without training datasets.

Contribution

The paper presents a novel neural disparity field architecture with an iterative reconstruction method that overcomes traditional limitations and does not require training datasets.

Findings

High-quality NDF can be reconstructed from light-field data.

NDF enables high-resolution disparity recovery.

Method works across various light-field acquisition methods.

Abstract

This study proposes a neural disparity field (NDF) that establishes an implicit, continuous representation of scene disparity based on a neural field and an iterative approach to address the inverse problem of NDF reconstruction from light-field data. NDF enables seamless and precise characterization of disparity variations in three-dimensional scenes and can discretize disparity at any arbitrary resolution, overcoming the limitations of traditional disparity maps that are prone to sampling errors and interpolation inaccuracies. The proposed NDF network architecture utilizes hash encoding combined with multilayer perceptrons to capture detailed disparities in texture levels, thereby enhancing its ability to represent the geometric information of complex scenes. By leveraging the spatial-angular consistency inherent in light-field data, a differentiable forward model to generate a central view image from the light-field data is developed. Based on the forward model, an optimization scheme for the inverse problem of NDF reconstruction using differentiable propagation operators is established. Furthermore, an iterative solution method is adopted to reconstruct the NDF in the optimization scheme, which does not require training datasets and applies to light-field data captured by various acquisition methods. Experimental results demonstrate that high-quality NDF can be reconstructed from light-field data using the proposed method. High-resolution disparity can be effectively recovered by NDF, demonstrating its capability for the implicit, continuous representation of scene disparities.