Pushing the boundaries of event subsampling in event-based video classification using CNNs

TL;DR

This paper investigates how reducing event data in event-based video classification affects CNN accuracy, revealing significant event rate reductions are possible with minimal accuracy loss and highlighting training challenges at high subsampling levels.

Contribution

It demonstrates that event subsampling can drastically reduce data without major accuracy loss and introduces a new metric to evaluate CNN hyperparameter sensitivity under subsampling.

Findings

Event rate can be reduced by an order of magnitude with little accuracy drop.

Training CNNs becomes more sensitive to hyperparameters at high subsampling rates.

A novel metric quantifies CNN training instability across datasets.

Abstract

Event cameras offer low-power visual sensing capabilities ideal for edge-device applications. However, their high event rate, driven by high temporal details, can be restrictive in terms of bandwidth and computational resources. In edge AI applications, determining the minimum amount of events for specific tasks can allow reducing the event rate to improve bandwidth, memory, and processing efficiency. In this paper, we study the effect of event subsampling on the accuracy of event data classification using convolutional neural network (CNN) models. Surprisingly, across various datasets, the number of events per video can be reduced by an order of magnitude with little drop in accuracy, revealing the extent to which we can push the boundaries in accuracy vs. event rate trade-off. Additionally, we also find that lower classification accuracy in high subsampling rates is not solely attributable to information loss due to the subsampling of the events, but that the training of CNNs can be challenging in highly subsampled scenarios, where the sensitivity to hyperparameters increases. We quantify training instability across multiple event-based classification datasets using a novel metric for evaluating the hyperparameter sensitivity of CNNs in different subsampling settings. Finally, we analyze the weight gradients of the network to gain insight into this instability.