ActiveZero: Mixed Domain Learning for Active Stereovision with Zero Annotation

TL;DR

ActiveZero is a mixed domain learning framework for active stereovision that achieves high accuracy without real-world depth annotations by combining simulation training, self-supervised learning, and a novel temporal IR reprojection loss.

Contribution

It introduces a new mixed domain learning approach with a novel self-supervised loss, enabling zero-annotation training for active stereovision systems.

Findings

Outperforms existing methods on real data

Achieves results comparable to or better than commercial depth sensors

Demonstrates effective transferability from simulation to real-world data

Abstract

Traditional depth sensors generate accurate real world depth estimates that surpass even the most advanced learning approaches trained only on simulation domains. Since ground truth depth is readily available in the simulation domain but quite difficult to obtain in the real domain, we propose a method that leverages the best of both worlds. In this paper we present a new framework, ActiveZero, which is a mixed domain learning solution for active stereovision systems that requires no real world depth annotation. First, we demonstrate the transferability of our method to out-of-distribution real data by using a mixed domain learning strategy. In the simulation domain, we use a combination of supervised disparity loss and self-supervised losses on a shape primitives dataset. By contrast, in the real domain, we only use self-supervised losses on a dataset that is out-of-distribution from either training simulation data or test real data. Second, our method introduces a novel self-supervised loss called temporal IR reprojection to increase the robustness and accuracy of our reprojections in hard-to-perceive regions. Finally, we show how the method can be trained end-to-end and that each module is important for attaining the end result. Extensive qualitative and quantitative evaluations on real data demonstrate state of the art results that can even beat a commercial depth sensor.