On-Device Self-Supervised Learning of Low-Latency Monocular Depth from Only Events

TL;DR

This paper presents an efficient on-device self-supervised learning pipeline for monocular depth estimation from event cameras, enabling resource-constrained robots like drones to learn and improve perception in real-time.

Contribution

It introduces a memory and time-efficient contrast maximization method for on-board learning, achieving state-of-the-art depth accuracy in resource-limited robotic platforms.

Findings

Online learning improves depth accuracy and obstacle avoidance.

The pipeline outperforms existing self-supervised methods in benchmarks.

Real-time on-device depth estimation is feasible with the proposed approach.

Abstract

Event cameras provide low-latency perception for only milliwatts of power. This makes them highly suitable for resource-restricted, agile robots such as small flying drones. Self-supervised learning based on contrast maximization holds great potential for event-based robot vision, as it foregoes the need for high-frequency ground truth and allows for online learning in the robot's operational environment. However, online, on-board learning raises the major challenge of achieving sufficient computational efficiency for real-time learning, while maintaining competitive visual perception performance. In this work, we improve the time and memory efficiency of the contrast maximization pipeline, making on-device learning of low-latency monocular depth possible. We demonstrate that online learning on board a small drone yields more accurate depth estimates and more successful obstacle avoidance behavior compared to only pre-training. Benchmarking experiments show that the proposed pipeline is not only efficient, but also achieves state-of-the-art depth estimation performance among self-supervised approaches. Our work taps into the unused potential of online, on-device robot learning, promising smaller reality gaps and better performance.