Semi-Supervised Adversarial Monocular Depth Estimation

TL;DR

This paper introduces a semi-supervised adversarial framework for monocular depth estimation that effectively leverages limited labeled data and abundant unlabeled images to improve accuracy and generalization.

Contribution

It proposes a novel semi-supervised adversarial learning method with dual discriminators for monocular depth estimation, reducing the need for large labeled datasets.

Findings

Improves state-of-the-art models on NYUD v2 with unlabeled data

Achieves state-of-the-art accuracy with small training sets on Make3D

Adapts well to unseen datasets after training on different data

Abstract

In this paper, we address the problem of monocular depth estimation when only a limited number of training image-depth pairs are available. To achieve a high regression accuracy, the state-of-the-art estimation methods rely on CNNs trained with a large number of image-depth pairs, which are prohibitively costly or even infeasible to acquire. Aiming to break the curse of such expensive data collections, we propose a semi-supervised adversarial learning framework that only utilizes a small number of image-depth pairs in conjunction with a large number of easily-available monocular images to achieve high performance. In particular, we use one generator to regress the depth and two discriminators to evaluate the predicted depth , i.e., one inspects the image-depth pair while the other inspects the depth channel alone. These two discriminators provide their feedbacks to the generator as the loss to generate more realistic and accurate depth predictions. Experiments show that the proposed approach can (1) improve most state-of-the-art models on the NYUD v2 dataset by effectively leveraging additional unlabeled data sources; (2) reach state-of-the-art accuracy when the training set is small, e.g., on the Make3D dataset; (3) adapt well to an unseen new dataset (Make3D in our case) after training on an annotated dataset (KITTI in our case).