Multi-scale Alternated Attention Transformer for Generalized Stereo Matching

TL;DR

This paper introduces AAUformer, a novel multi-scale alternated attention transformer network that enhances stereo matching by balancing intra- and inter-view features, achieving state-of-the-art results and strong generalization.

Contribution

The paper proposes a new AAUformer architecture with window self-attention and multi-scale alternated attention for improved stereo matching and generalization.

Findings

Achieves state-of-the-art performance on Scene Flow dataset.

Performs competitively on KITTI 2015 after fine-tuning.

Outperforms existing methods in cross-dataset generalization.

Abstract

Recent stereo matching networks achieves dramatic performance by introducing epipolar line constraint to limit the matching range of dual-view. However, in complicated real-world scenarios, the feature information based on intra-epipolar line alone is too weak to facilitate stereo matching. In this paper, we present a simple but highly effective network called Alternated Attention U-shaped Transformer (AAUformer) to balance the impact of epipolar line in dual and single view respectively for excellent generalization performance. Compared to other models, our model has several main designs: 1) to better liberate the local semantic features of the single-view at pixel level, we introduce window self-attention to break the limits of intra-row self-attention and completely replace the convolutional network for denser features before cross-matching; 2) the multi-scale alternated attention backbone network was designed to extract invariant features in order to achieves the coarse-to-fine matching process for hard-to-discriminate regions. We performed a series of both comparative studies and ablation studies on several mainstream stereo matching datasets. The results demonstrate that our model achieves state-of-the-art on the Scene Flow dataset, and the fine-tuning performance is competitive on the KITTI 2015 dataset. In addition, for cross generalization experiments on synthetic and real-world datasets, our model outperforms several state-of-the-art works.