Context-Enhanced Stereo Transformer

TL;DR

This paper introduces Context Enhanced Stereo Transformer (CSTR), a novel model that incorporates a Context Enhanced Path to improve stereo depth estimation, especially in challenging regions, by capturing long-range global information, leading to superior performance across multiple datasets.

Contribution

The paper proposes the CEP module integrated into a stereo transformer to enhance generalization and robustness in stereo depth estimation, addressing limitations of existing methods.

Findings

CSTR outperforms prior approaches on multiple datasets.

CEP effectively captures long-range global information.

CSTR achieves an 11% improvement in zero-shot synthetic-to-real transfer on Middlebury-2014.

Abstract

Stereo depth estimation is of great interest for computer vision research. However, existing methods struggles to generalize and predict reliably in hazardous regions, such as large uniform regions. To overcome these limitations, we propose Context Enhanced Path (CEP). CEP improves the generalization and robustness against common failure cases in existing solutions by capturing the long-range global information. We construct our stereo depth estimation model, Context Enhanced Stereo Transformer (CSTR), by plugging CEP into the state-of-the-art stereo depth estimation method Stereo Transformer. CSTR is examined on distinct public datasets, such as Scene Flow, Middlebury-2014, KITTI-2015, and MPI-Sintel. We find CSTR outperforms prior approaches by a large margin. For example, in the zero-shot synthetic-to-real setting, CSTR outperforms the best competing approaches on Middlebury-2014 dataset by 11%. Our extensive experiments demonstrate that the long-range information is critical for stereo matching task and CEP successfully captures such information.