LiteDepth: Digging into Fast and Accurate Depth Estimation on Mobile Devices

TL;DR

LiteDepth presents a lightweight, fast, and accurate monocular depth estimation model optimized for mobile devices, achieving real-time inference with minimal size and high performance through novel training strategies and distillation.

Contribution

The paper introduces a compact depth estimation model with innovative data augmentation, multi-loss training, dynamic re-weighting, and structure-aware distillation for mobile deployment.

Findings

Achieves 37ms inference on Raspberry Pi 4

Ranks 2nd in MAI&AIM2022 challenge

Provides the fastest solution with high accuracy

Abstract

Monocular depth estimation is an essential task in the computer vision community. While tremendous successful methods have obtained excellent results, most of them are computationally expensive and not applicable for real-time on-device inference. In this paper, we aim to address more practical applications of monocular depth estimation, where the solution should consider not only the precision but also the inference time on mobile devices. To this end, we first develop an end-to-end learning-based model with a tiny weight size (1.4MB) and a short inference time (27FPS on Raspberry Pi 4). Then, we propose a simple yet effective data augmentation strategy, called R2 crop, to boost the model performance. Moreover, we observe that the simple lightweight model trained with only one single loss term will suffer from performance bottleneck. To alleviate this issue, we adopt multiple loss terms to provide sufficient constraints during the training stage. Furthermore, with a simple dynamic re-weight strategy, we can avoid the time-consuming hyper-parameter choice of loss terms. Finally, we adopt the structure-aware distillation to further improve the model performance. Notably, our solution named LiteDepth ranks 2nd in the MAI&AIM2022 Monocular Depth Estimation Challenge}, with a si-RMSE of 0.311, an RMSE of 3.79, and the inference time is 37$ms$ tested on the Raspberry Pi 4. Notably, we provide the fastest solution to the challenge. Codes and models will be released at \url{https://github.com/zhyever/LiteDepth}.