Towards Adversarially Robust and Domain Generalizable Stereo Matching by Rethinking DNN Feature Backbones

TL;DR

This paper addresses the vulnerability of DNN-based stereo matching to adversarial attacks and poor domain generalization, proposing a method that removes learnable feature backbones and uses classic image transforms to improve robustness and transferability.

Contribution

It introduces a novel stereo matching approach that eliminates learnable feature backbones, replacing them with classic transforms, enhancing adversarial robustness and domain generalization.

Findings

Significantly improves adversarial robustness against white-box attacks.

Achieves comparable accuracy to state-of-the-art methods on benchmark datasets.

Demonstrates better transferability from simulation to real-world data without fine-tuning.

Abstract

Stereo matching has recently witnessed remarkable progress using Deep Neural Networks (DNNs). But, how robust are they? Although it has been well-known that DNNs often suffer from adversarial vulnerability with a catastrophic drop in performance, the situation is even worse in stereo matching. This paper first shows that a type of weak white-box attacks can overwhelm state-of-the-art methods. The attack is learned by a proposed stereo-constrained projected gradient descent (PGD) method in stereo matching. This observation raises serious concerns for the deployment of DNN-based stereo matching. Parallel to the adversarial vulnerability, DNN-based stereo matching is typically trained under the so-called simulation to reality pipeline, and thus domain generalizability is an important problem. This paper proposes to rethink the learnable DNN-based feature backbone towards adversarially-robust and domain generalizable stereo matching by completely removing it for matching. In experiments, the proposed method is tested in the SceneFlow dataset and the KITTI2015 benchmark, with promising results. We compute the matching cost volume using the classic multi-scale census transform (i.e., local binary pattern) of the raw input stereo images, followed by a stacked Hourglass head sub-network solving the matching problem. It significantly improves the adversarial robustness, while retaining accuracy performance comparable to state-of-the-art methods. It also shows better generalizability from simulation (SceneFlow) to real (KITTI) datasets when no fine-tuning is used.