Selective-Stereo: Adaptive Frequency Information Selection for Stereo Matching

TL;DR

Selective-Stereo introduces an adaptive frequency information selection mechanism via the SRU module, enhancing iterative stereo matching by effectively capturing details in edges and smooth regions, leading to state-of-the-art results.

Contribution

The paper proposes the SRU module and CSA attention mechanism, enabling adaptive fusion of multi-frequency disparity information in iterative stereo matching methods.

Findings

Achieves first place on KITTI 2012, KITTI 2015, ETH3D, and Middlebury benchmarks.

Demonstrates improved detail preservation and edge accuracy.

Validates SRU's effectiveness across multiple stereo matching frameworks.

Abstract

Stereo matching methods based on iterative optimization, like RAFT-Stereo and IGEV-Stereo, have evolved into a cornerstone in the field of stereo matching. However, these methods struggle to simultaneously capture high-frequency information in edges and low-frequency information in smooth regions due to the fixed receptive field. As a result, they tend to lose details, blur edges, and produce false matches in textureless areas. In this paper, we propose Selective Recurrent Unit (SRU), a novel iterative update operator for stereo matching. The SRU module can adaptively fuse hidden disparity information at multiple frequencies for edge and smooth regions. To perform adaptive fusion, we introduce a new Contextual Spatial Attention (CSA) module to generate attention maps as fusion weights. The SRU empowers the network to aggregate hidden disparity information across multiple frequencies, mitigating the risk of vital hidden disparity information loss during iterative processes. To verify SRU's universality, we apply it to representative iterative stereo matching methods, collectively referred to as Selective-Stereo. Our Selective-Stereo ranks $1^{st}$ on KITTI 2012, KITTI 2015, ETH3D, and Middlebury leaderboards among all published methods. Code is available at https://github.com/Windsrain/Selective-Stereo.