Self-Supervised Depth Estimation with Isometric-Self-Sample-Based Learning

TL;DR

This paper introduces ISSL, a novel self-supervised learning method for depth estimation that synthesizes static scene-consistent self-samples to improve training efficiency and accuracy across various datasets.

Contribution

The paper proposes ISSL, a simple yet effective self-sample generation technique that enhances self-supervised depth estimation by fully utilizing training images.

Findings

Consistently improves depth estimation performance when integrated into existing models.

Enhances accuracy across outdoor and indoor scene datasets.

Significantly boosts depth estimation results on KITTI, Make3D, and NYUv2.

Abstract

Managing the dynamic regions in the photometric loss formulation has been a main issue for handling the self-supervised depth estimation problem. Most previous methods have alleviated this issue by removing the dynamic regions in the photometric loss formulation based on the masks estimated from another module, making it difficult to fully utilize the training images. In this paper, to handle this problem, we propose an isometric self-sample-based learning (ISSL) method to fully utilize the training images in a simple yet effective way. The proposed method provides additional supervision during training using self-generated images that comply with pure static scene assumption. Specifically, the isometric self-sample generator synthesizes self-samples for each training image by applying random rigid transformations on the estimated depth. Thus both the generated self-samples and the corresponding training image always follow the static scene assumption. We show that plugging our ISSL module into several existing models consistently improves the performance by a large margin. In addition, it also boosts the depth accuracy over different types of scene, i.e., outdoor scenes (KITTI and Make3D) and indoor scene (NYUv2), validating its high effectiveness.