Self-Supervised Learning for Stereo Matching with Self-Improving Ability

TL;DR

This paper introduces a self-supervised deep learning approach for stereo matching that learns disparity maps directly from stereo images without ground-truth data, using image warping error as the loss, achieving superior speed and accuracy.

Contribution

The authors propose a novel self-supervised neural network architecture for stereo matching that eliminates the need for ground-truth disparity maps during training.

Findings

Outperforms state-of-the-art methods on KITTI and Middlebury datasets.

Achieves faster inference while maintaining high accuracy.

Demonstrates adaptability to different camera settings and unseen images.

Abstract

Exiting deep-learning based dense stereo matching methods often rely on ground-truth disparity maps as the training signals, which are however not always available in many situations. In this paper, we design a simple convolutional neural network architecture that is able to learn to compute dense disparity maps directly from the stereo inputs. Training is performed in an end-to-end fashion without the need of ground-truth disparity maps. The idea is to use image warping error (instead of disparity-map residuals) as the loss function to drive the learning process, aiming to find a depth-map that minimizes the warping error. While this is a simple concept well-known in stereo matching, to make it work in a deep-learning framework, many non-trivial challenges must be overcome, and in this work we provide effective solutions. Our network is self-adaptive to different unseen imageries as well as to different camera settings. Experiments on KITTI and Middlebury stereo benchmark datasets show that our method outperforms many state-of-the-art stereo matching methods with a margin, and at the same time significantly faster.