Matching-space Stereo Networks for Cross-domain Generalization

TL;DR

This paper introduces Matching-Space Networks (MS-Nets), a novel stereo matching architecture that enhances cross-domain generalization by replacing learned features with traditional matching functions, achieving superior performance on unseen datasets.

Contribution

The paper proposes MS-Nets, a new architecture that shifts from RGB feature learning to matching functions, improving generalization across different domains in stereo matching tasks.

Findings

MS-Nets outperform traditional deep stereo networks on unseen datasets.

Accuracy on the source domain remains nearly unchanged with MS-Nets.

Extensive experiments validate the improved cross-domain generalization of MS-Nets.

Abstract

End-to-end deep networks represent the state of the art for stereo matching. While excelling on images framing environments similar to the training set, major drops in accuracy occur in unseen domains (e.g., when moving from synthetic to real scenes). In this paper we introduce a novel family of architectures, namely Matching-Space Networks (MS-Nets), with improved generalization properties. By replacing learning-based feature extraction from image RGB values with matching functions and confidence measures from conventional wisdom, we move the learning process from the color space to the Matching Space, avoiding over-specialization to domain specific features. Extensive experimental results on four real datasets highlight that our proposal leads to superior generalization to unseen environments over conventional deep architectures, keeping accuracy on the source domain almost unaltered. Our code is available at https://github.com/ccj5351/MS-Nets.