SAFENet: Self-Supervised Monocular Depth Estimation with Semantic-Aware Feature Extraction

TL;DR

SAFENet improves self-supervised monocular depth estimation by integrating semantic-aware features, leading to more accurate, robust, and generalizable depth predictions without groundtruth depth maps.

Contribution

The paper introduces a novel multi-task learning approach that incorporates semantic information into depth features for self-supervised monocular depth estimation.

Findings

Outperforms state-of-the-art methods on KITTI dataset

Demonstrates better generalization across different datasets

Shows robustness under low-light and adverse weather conditions

Abstract

Self-supervised monocular depth estimation has emerged as a promising method because it does not require groundtruth depth maps during training. As an alternative for the groundtruth depth map, the photometric loss enables to provide self-supervision on depth prediction by matching the input image frames. However, the photometric loss causes various problems, resulting in less accurate depth values compared with supervised approaches. In this paper, we propose SAFENet that is designed to leverage semantic information to overcome the limitations of the photometric loss. Our key idea is to exploit semantic-aware depth features that integrate the semantic and geometric knowledge. Therefore, we introduce multi-task learning schemes to incorporate semantic-awareness into the representation of depth features. Experiments on KITTI dataset demonstrate that our methods compete or even outperform the state-of-the-art methods. Furthermore, extensive experiments on different datasets show its better generalization ability and robustness to various conditions, such as low-light or adverse weather.