Adversarial View-Consistent Learning for Monocular Depth Estimation

TL;DR

This paper introduces an adversarial learning framework that enforces view consistency in monocular depth estimation, improving the geometric accuracy of depth maps across multiple viewpoints.

Contribution

It proposes a novel adversarial view-consistent learning framework with a differentiable warping operation and pose generator for better multi-view depth estimation.

Findings

Improved depth estimation accuracy on NYU Depth V2 dataset.

Enhanced view consistency in predicted depth maps.

Outperforms state-of-the-art MDE methods.

Abstract

This paper addresses the problem of Monocular Depth Estimation (MDE). Existing approaches on MDE usually model it as a pixel-level regression problem, ignoring the underlying geometry property. We empirically find this may result in sub-optimal solution: while the predicted depth map presents small loss value in one specific view, it may exhibit large loss if viewed in different directions. In this paper, inspired by multi-view stereo (MVS), we propose an Adversarial View-Consistent Learning (AVCL) framework to force the estimated depth map to be all reasonable viewed from multiple views. To this end, we first design a differentiable depth map warping operation, which is end-to-end trainable, and then propose a pose generator to generate novel views for a given image in an adversarial manner. Collaborating with the differentiable depth map warping operation, the pose generator encourages the depth estimation network to learn from hard views, hence produce view-consistent depth maps . We evaluate our method on NYU Depth V2 dataset and the experimental results show promising performance gain upon state-of-the-art MDE approaches.