Geometry-Aware Unsupervised Domain Adaptation for Stereo Matching

TL;DR

This paper introduces a geometry-aware unsupervised domain adaptation method for stereo matching that uses a novel attention mechanism to preserve the geometric structure during image translation, improving accuracy across environments.

Contribution

It proposes Stereoscopic Cross Attention (SCA), an attention mechanism that maintains stereo geometry during image-to-image translation for domain adaptation.

Findings

Effective preservation of stereo geometry in unsupervised adaptation

Improved stereo matching accuracy in new environments

Demonstrated superiority over existing translation-based methods

Abstract

Recently proposed DNN-based stereo matching methods that learn priors directly from data are known to suffer a drastic drop in accuracy in new environments. Although supervised approaches with ground truth disparity maps often work well, collecting them in each deployment environment is cumbersome and costly. For this reason, many unsupervised domain adaptation methods based on image-to-image translation have been proposed, but these methods do not preserve the geometric structure of a stereo image pair because the image-to-image translation is applied to each view separately. To address this problem, in this paper, we propose an attention mechanism that aggregates features in the left and right views, called Stereoscopic Cross Attention (SCA). Incorporating SCA to an image-to-image translation network makes it possible to preserve the geometric structure of a stereo image pair in the process of the image-to-image translation. We empirically demonstrate the effectiveness of the proposed unsupervised domain adaptation based on the image-to-image translation with SCA.