Degradation-agnostic Correspondence from Resolution-asymmetric Stereo

TL;DR

This paper introduces a novel unsupervised stereo matching approach that leverages feature-metric consistency to handle resolution asymmetry and unknown degradations, improving disparity estimation in real-world scenarios.

Contribution

It proposes a degradation-agnostic feature-metric loss and a self-boosting strategy to enhance stereo matching from resolution-asymmetric images without ground-truth labels.

Findings

Outperforms existing methods on simulated datasets with various degradations.

Achieves superior results on real-world stereo datasets.

Demonstrates robustness to unknown image degradations.

Abstract

In this paper, we study the problem of stereo matching from a pair of images with different resolutions, e.g., those acquired with a tele-wide camera system. Due to the difficulty of obtaining ground-truth disparity labels in diverse real-world systems, we start from an unsupervised learning perspective. However, resolution asymmetry caused by unknown degradations between two views hinders the effectiveness of the generally assumed photometric consistency. To overcome this challenge, we propose to impose the consistency between two views in a feature space instead of the image space, named feature-metric consistency. Interestingly, we find that, although a stereo matching network trained with the photometric loss is not optimal, its feature extractor can produce degradation-agnostic and matching-specific features. These features can then be utilized to formulate a feature-metric loss to avoid the photometric inconsistency. Moreover, we introduce a self-boosting strategy to optimize the feature extractor progressively, which further strengthens the feature-metric consistency. Experiments on both simulated datasets with various degradations and a self-collected real-world dataset validate the superior performance of the proposed method over existing solutions.