Improving Domain Generalization in Self-supervised Monocular Depth Estimation via Stabilized Adversarial Training

TL;DR

This paper introduces SCAT, a novel adversarial training framework that stabilizes self-supervised monocular depth estimation, significantly enhancing its generalization across diverse benchmarks.

Contribution

The paper proposes a conflict-optimized adversarial training method with a scaled depth network and gradient surgery to improve self-supervised MDE generalization.

Findings

SCAT achieves state-of-the-art results on five benchmarks.

It effectively balances stability and generalization in adversarial training.

The method outperforms existing self-supervised MDE approaches.

Abstract

Learning a self-supervised Monocular Depth Estimation (MDE) model with great generalization remains significantly challenging. Despite the success of adversarial augmentation in the supervised learning generalization, naively incorporating it into self-supervised MDE models potentially causes over-regularization, suffering from severe performance degradation. In this paper, we conduct qualitative analysis and illuminate the main causes: (i) inherent sensitivity in the UNet-alike depth network and (ii) dual optimization conflict caused by over-regularization. To tackle these issues, we propose a general adversarial training framework, named Stabilized Conflict-optimization Adversarial Training (SCAT), integrating adversarial data augmentation into self-supervised MDE methods to achieve a balance between stability and generalization. Specifically, we devise an effective scaling depth network that tunes the coefficients of long skip connection and effectively stabilizes the training process. Then, we propose a conflict gradient surgery strategy, which progressively integrates the adversarial gradient and optimizes the model toward a conflict-free direction. Extensive experiments on five benchmarks demonstrate that SCAT can achieve state-of-the-art performance and significantly improve the generalization capability of existing self-supervised MDE methods.