Semantics-Driven Unsupervised Learning for Monocular Depth and Ego-Motion Estimation

TL;DR

This paper introduces a semantics-driven unsupervised learning method for monocular depth and ego-motion estimation that leverages semantic segmentation to improve accuracy and robustness without requiring manual labels.

Contribution

It utilizes semantic segmentation information and 3D spatial constraints to enhance depth and ego-motion estimation in an unsupervised manner, reducing reliance on labeled data.

Findings

Achieves competitive performance on KITTI dataset

Effectively handles dynamic objects and occlusions

Improves depth prediction accuracy

Abstract

We propose a semantics-driven unsupervised learning approach for monocular depth and ego-motion estimation from videos in this paper. Recent unsupervised learning methods employ photometric errors between synthetic view and actual image as a supervision signal for training. In our method, we exploit semantic segmentation information to mitigate the effects of dynamic objects and occlusions in the scene, and to improve depth prediction performance by considering the correlation between depth and semantics. To avoid costly labeling process, we use noisy semantic segmentation results obtained by a pre-trained semantic segmentation network. In addition, we minimize the position error between the corresponding points of adjacent frames to utilize 3D spatial information. Experimental results on the KITTI dataset show that our method achieves good performance in both depth and ego-motion estimation tasks.