Dehazing Cost Volume for Deep Multi-view Stereo in Scattering Media with Airlight and Scattering Coefficient Estimation

TL;DR

This paper introduces a novel dehazing cost volume for multi-view stereo in scattering media, enabling simultaneous depth estimation and image restoration by modeling scattering effects and estimating scattering parameters.

Contribution

It presents a new cost volume that accounts for scattering effects, along with a method to estimate scattering parameters, improving 3D reconstruction in foggy or smoky environments.

Findings

Outperforms traditional cost volumes in hazy image reconstruction

Effectively estimates scattering parameters like airlight and scattering coefficient

Successfully applied to real foggy scenes

Abstract

We propose a learning-based multi-view stereo (MVS) method in scattering media, such as fog or smoke, with a novel cost volume, called the dehazing cost volume. Images captured in scattering media are degraded due to light scattering and attenuation caused by suspended particles. This degradation depends on scene depth; thus, it is difficult for traditional MVS methods to evaluate photometric consistency because the depth is unknown before three-dimensional (3D) reconstruction. The dehazing cost volume can solve this chicken-and-egg problem of depth estimation and image restoration by computing the scattering effect using swept planes in the cost volume. We also propose a method of estimating scattering parameters, such as airlight, and a scattering coefficient, which are required for our dehazing cost volume. The output depth of a network with our dehazing cost volume can be regarded as a function of these parameters; thus, they are geometrically optimized with a sparse 3D point cloud obtained at a structure-from-motion step. Experimental results on synthesized hazy images indicate the effectiveness of our dehazing cost volume against the ordinary cost volume regarding scattering media. We also demonstrated the applicability of our dehazing cost volume to real foggy scenes.