FSNet: Redesign Self-Supervised MonoDepth for Full-Scale Depth Prediction for Autonomous Driving

TL;DR

FSNet is a novel self-supervised framework that achieves full-scale, accurate depth prediction for autonomous driving using monocular images, inertial data, and innovative training and post-processing techniques.

Contribution

Introduces FSNet, a full-scale depth prediction network with multichannel output, optical-flow-based dynamic object removal, self-distillation training, and test-time post-processing.

Findings

Achieves accurate depth estimation on KITTI, KITTI-360, and nuScenes datasets.

Outperforms existing self-supervised models in autonomous driving scenarios.

Enables monocular depth prediction without extra labeling or 3D data.

Abstract

Predicting accurate depth with monocular images is important for low-cost robotic applications and autonomous driving. This study proposes a comprehensive self-supervised framework for accurate scale-aware depth prediction on autonomous driving scenes utilizing inter-frame poses obtained from inertial measurements. In particular, we introduce a Full-Scale depth prediction network named FSNet. FSNet contains four important improvements over existing self-supervised models: (1) a multichannel output representation for stable training of depth prediction in driving scenarios, (2) an optical-flow-based mask designed for dynamic object removal, (3) a self-distillation training strategy to augment the training process, and (4) an optimization-based post-processing algorithm in test time, fusing the results from visual odometry. With this framework, robots and vehicles with only one well-calibrated camera can collect sequences of training image frames and camera poses, and infer accurate 3D depths of the environment without extra labeling work or 3D data. Extensive experiments on the KITTI dataset, KITTI-360 dataset and the nuScenes dataset demonstrate the potential of FSNet. More visualizations are presented in \url{https://sites.google.com/view/fsnet/home}