Object Motion Sensitivity: A Bio-inspired Solution to the Ego-motion Problem for Event-based Cameras

TL;DR

This paper demonstrates that bio-inspired neuromorphic sensors, specifically IRIS, can effectively perform object motion sensitivity tasks with high efficiency and significantly reduced bandwidth, enabling energy-efficient real-time vision applications.

Contribution

The work introduces the IRIS sensor's capability to process object motion sensitivity locally, achieving comparable performance to traditional sensors with much lower bandwidth.

Findings

IRIS sensor achieves similar object motion sensitivity as RGB and DVS sensors.

Significant bandwidth reduction leads to lower power and wireless communication costs.

Enables high-speed, energy-efficient, real-time decision making in vision systems.

Abstract

Neuromorphic (event-based) image sensors draw inspiration from the human-retina to create an electronic device that can process visual stimuli in a way that closely resembles its biological counterpart. These sensors process information significantly different than the traditional RGB sensors. Specifically, the sensory information generated by event-based image sensors are orders of magnitude sparser compared to that of RGB sensors. The first generation of neuromorphic image sensors, Dynamic Vision Sensor (DVS), are inspired by the computations confined to the photoreceptors and the first retinal synapse. In this work, we highlight the capability of the second generation of neuromorphic image sensors, Integrated Retinal Functionality in CMOS Image Sensors (IRIS), which aims to mimic full retinal computations from photoreceptors to output of the retina (retinal ganglion cells) for targeted feature-extraction. The feature of choice in this work is Object Motion Sensitivity (OMS) that is processed locally in the IRIS sensor. Our results show that OMS can accomplish standard computer vision tasks with similar efficiency to conventional RGB and DVS solutions but offers drastic bandwidth reduction. This cuts the wireless and computing power budgets and opens up vast opportunities in high-speed, robust, energy-efficient, and low-bandwidth real-time decision making.