BlinkBud: Detecting Hazards from Behind via Sampled Monocular 3D Detection on a Single Earbud

TL;DR

BlinkBud is a lightweight, low-power system using a single earbud and phone to detect hazards from behind by tracking objects with a novel 3D detection and sampling strategy, improving safety for pedestrians and cyclists.

Contribution

The paper introduces BlinkBud, a novel system combining monocular 3D detection, reinforcement learning-based sampling, and head movement compensation for hazard detection from behind.

Findings

Achieves low false positive and false negative ratios.

Consumes ultra-low power on earbud and phone.

Successfully detects hazards in real-world tests.

Abstract

Failing to be aware of speeding vehicles approaching from behind poses a huge threat to the road safety of pedestrians and cyclists. In this paper, we propose BlinkBud, which utilizes a single earbud and a paired phone to online detect hazardous objects approaching from behind of a user. The core idea is to accurately track visually identified objects utilizing a small number of sampled camera images taken from the earbud. To minimize the power consumption of the earbud and the phone while guaranteeing the best tracking accuracy, a novel 3D object tracking algorithm is devised, integrating both a Kalman filter based trajectory estimation scheme and an optimal image sampling strategy based on reinforcement learning. Moreover, the impact of constant user head movements on the tracking accuracy is significantly eliminated by leveraging the estimated pitch and yaw angles to correct the object depth estimation and align the camera coordinate system to the user's body coordinate system, respectively. We implement a prototype BlinkBud system and conduct extensive real-world experiments. Results show that BlinkBud is lightweight with ultra-low mean power consumptions of 29.8 mW and 702.6 mW on the earbud and smartphone, respectively, and can accurately detect hazards with a low average false positive ratio (FPR) and false negative ratio (FNR) of 4.90% and 1.47%, respectively.