See Silhouettes in Motion with Neuromorphic Vision

TL;DR

This paper introduces a dual-modal approach combining frame and event data for real-time binarization, improving performance in dynamic and challenging lighting conditions on resource-limited devices.

Contribution

It presents a novel, effective method leveraging event cameras and frames for high-speed binarization, addressing motion blur and illumination issues in embedded vision systems.

Findings

Achieves real-time, high-frame-rate binarization on CPU-only devices.

Reduces motion blur and improves performance under harsh lighting.

Maintains clear silhouettes even at kilohertz frame rates.

Abstract

Quasi-bimodal objects, such as text, road signs, and barcodes, play a basic yet vital role in daily visual communication. By boiling these down to clear silhouettes, binarization uses a minimal language to convey essential vision cues for maximum downstream efficiency. The catch is that frame-based imaging often struggles on mobile platforms like drones, self-driving cars, and underwater vehicles. In these dynamic scenes, rapid motion and harsh lighting can make it blind, causing severe motion blur and erasing crucial details. To overcome the limits, neuromorphic vision via event cameras, featuring microsecond-level temporal resolution and high dynamic range, steps in as a natural solution. Building upon this event-driven sensing paradigm, we introduce a simple yet effective dual-modal approach that harnesses the synergy between frames and events to achieve real-time, high-frame-rate binarization on CPU-only devices. Extensive evaluations present that it earns competitive performance against leading techniques in reducing motion blur, while delivering impressive improvements under challenging illumination. Besides, our asynchronous workflow bypasses event scarcity that breaks traditional time-binning reconstruction, maintaining clear target shapes even at extreme kilohertz frame rates. Its binary results further serve as reliable representations that facilitate a range of downstream tasks. This work paves the way towards lightweight perception and interaction in embodied intelligence on resource-constrained edge platforms.