Investigation of event-based memory surfaces for high-speed tracking, unsupervised feature extraction and object recognition

TL;DR

This study compares event-based decaying and time-based memory surfaces for high-speed object tracking and recognition, demonstrating that decaying surfaces, especially with exponential decay, improve velocity invariance and recognition accuracy.

Contribution

It introduces and evaluates event-based decaying memory surfaces, showing their superiority over time-based surfaces for high-speed event-based object recognition.

Findings

Decaying memory surfaces outperform time-based surfaces in recognition accuracy.

Exponential decay kernels yield the best performance among tested methods.

System achieves 98.75% accuracy within 156 ms using minimal features.

Abstract

In this paper we compare event-based decaying and time based-decaying memory surfaces for high-speed eventbased tracking, feature extraction, and object classification using an event-based camera. The high-speed recognition task involves detecting and classifying model airplanes that are dropped free-hand close to the camera lens so as to generate a challenging dataset exhibiting significant variance in target velocity. This variance motivated the investigation of event-based decaying memory surfaces in comparison to time-based decaying memory surfaces to capture the temporal aspect of the event-based data. These surfaces are then used to perform unsupervised feature extraction, tracking and recognition. In order to generate the memory surfaces, event binning, linearly decaying kernels, and exponentially decaying kernels were investigated with exponentially decaying kernels found to perform best. Event-based decaying memory surfaces were found to outperform time-based decaying memory surfaces in recognition especially when invariance to target velocity was made a requirement. A range of network and receptive field sizes were investigated. The system achieves 98.75% recognition accuracy within 156 milliseconds of an airplane entering the field of view, using only twenty-five event-based feature extracting neurons in series with a linear classifier. By comparing the linear classifier results to an ELM classifier, we find that a small number of event-based feature extractors can effectively project the complex spatio-temporal event patterns of the dataset to an almost linearly separable representation in feature space.