DCL: Differential Contrastive Learning for Geometry-Aware Depth Synthesis

TL;DR

This paper introduces a novel differential contrastive learning approach for unpaired, geometry-aware depth synthesis that improves the realism and diversity of synthetic depth data for training task-specific networks.

Contribution

It presents a new depth synthesis method that explicitly preserves geometric properties using differential contrastive learning, outperforming existing methods in various geometric reasoning tasks.

Findings

Synthesized depth improves task performance over existing methods.

Networks trained with our depth data surpass fully supervised models after fine-tuning.

The method is effective across multiple real-world geometric reasoning tasks.

Abstract

We describe a method for unpaired realistic depth synthesis that learns diverse variations from the real-world depth scans and ensures geometric consistency between the synthetic and synthesized depth. The synthesized realistic depth can then be used to train task-specific networks facilitating label transfer from the synthetic domain. Unlike existing image synthesis pipelines, where geometries are mostly ignored, we treat geometries carried by the depth scans based on their own existence. We propose differential contrastive learning that explicitly enforces the underlying geometric properties to be invariant regarding the real variations been learned. The resulting depth synthesis method is task-agnostic, and we demonstrate the effectiveness of the proposed synthesis method by extensive evaluations on real-world geometric reasoning tasks. The networks trained with the depth synthesized by our method consistently achieve better performance across a wide range of tasks than state of the art, and can even surpass the networks supervised with full real-world annotations when slightly fine-tuned, showing good transferability.