Movement-induced Priors for Deep Stereo

TL;DR

This paper introduces a novel approach that integrates movement-induced priors into deep stereo disparity estimation using Gaussian process kernels, enhancing performance especially with low-quality sensor data.

Contribution

It presents a new movement-driven Gaussian process kernel for stereo depth estimation that works with limited motion information and can be integrated with existing deep stereo networks.

Findings

Improves stereo disparity estimation by incorporating movement priors.

Compatible with pre-trained deep stereo models for plug-and-play enhancement.

Joint training of kernels and networks further boosts accuracy.

Abstract

We propose a method for fusing stereo disparity estimation with movement-induced prior information. Instead of independent inference frame-by-frame, we formulate the problem as a non-parametric learning task in terms of a temporal Gaussian process prior with a movement-driven kernel for inter-frame reasoning. We present a hierarchy of three Gaussian process kernels depending on the availability of motion information, where our main focus is on a new gyroscope-driven kernel for handheld devices with low-quality MEMS sensors, thus also relaxing the requirement of having full 6D camera poses available. We show how our method can be combined with two state-of-the-art deep stereo methods. The method either work in a plug-and-play fashion with pre-trained deep stereo networks, or further improved by jointly training the kernels together with encoder-decoder architectures, leading to consistent improvement.