Real-time single image depth perception in the wild with handheld devices

TL;DR

This paper presents a new approach for real-time depth perception from a single image on handheld devices, addressing reliability and resource challenges to enable practical deployment in diverse real-world scenarios.

Contribution

It introduces network design and training strategies that improve in-the-wild reliability and efficiency, facilitating real-time depth estimation on standard handheld devices.

Findings

Fast networks generalize well to new environments

Achieves real-time performance on smartphones

Effective for depth-aware augmented reality and image blurring

Abstract

Depth perception is paramount to tackle real-world problems, ranging from autonomous driving to consumer applications. For the latter, depth estimation from a single image represents the most versatile solution, since a standard camera is available on almost any handheld device. Nonetheless, two main issues limit its practical deployment: i) the low reliability when deployed in-the-wild and ii) the demanding resource requirements to achieve real-time performance, often not compatible with such devices. Therefore, in this paper, we deeply investigate these issues showing how they are both addressable adopting appropriate network design and training strategies -- also outlining how to map the resulting networks on handheld devices to achieve real-time performance. Our thorough evaluation highlights the ability of such fast networks to generalize well to new environments, a crucial feature required to tackle the extremely varied contexts faced in real applications. Indeed, to further support this evidence, we report experimental results concerning real-time depth-aware augmented reality and image blurring with smartphones in-the-wild.