StereoFlowGAN: Co-training for Stereo and Flow with Unsupervised Domain Adaptation

TL;DR

StereoFlowGAN presents a new unsupervised domain adaptation framework that improves stereo matching and optical flow estimation by leveraging image translation between synthetic and real images, using a novel bidirectional feature warping module.

Contribution

It introduces a task-agnostic domain adaptation method with a bidirectional feature warping module for stereo and flow tasks, enabling effective training with synthetic ground-truth data.

Findings

Achieves competitive performance on real image datasets.

Effectively leverages synthetic data for real-world applications.

Demonstrates the benefits of unsupervised domain adaptation in stereo and flow estimation.

Abstract

We introduce a novel training strategy for stereo matching and optical flow estimation that utilizes image-to-image translation between synthetic and real image domains. Our approach enables the training of models that excel in real image scenarios while relying solely on ground-truth information from synthetic images. To facilitate task-agnostic domain adaptation and the training of task-specific components, we introduce a bidirectional feature warping module that handles both left-right and forward-backward directions. Experimental results show competitive performance over previous domain translation-based methods, which substantiate the efficacy of our proposed framework, effectively leveraging the benefits of unsupervised domain adaptation, stereo matching, and optical flow estimation.