Event Neural Networks

TL;DR

Event Neural Networks (EvNets) exploit video redundancy and neuron state memory to significantly reduce computational costs during inference, maintaining accuracy across various vision tasks without retraining.

Contribution

EvNets introduce a novel neuron state mechanism enabling transformation of existing networks into efficient, low-cost models without retraining, applicable to multiple visual processing tasks.

Findings

Approximately tenfold reduction in computation costs.

Minimal accuracy loss across tasks.

Applicable to diverse neural network architectures.

Abstract

Video data is often repetitive; for example, the contents of adjacent frames are usually strongly correlated. Such redundancy occurs at multiple levels of complexity, from low-level pixel values to textures and high-level semantics. We propose Event Neural Networks (EvNets), which leverage this redundancy to achieve considerable computation savings during video inference. A defining characteristic of EvNets is that each neuron has state variables that provide it with long-term memory, which allows low-cost, high-accuracy inference even in the presence of significant camera motion. We show that it is possible to transform a wide range of neural networks into EvNets without re-training. We demonstrate our method on state-of-the-art architectures for both high- and low-level visual processing, including pose recognition, object detection, optical flow, and image enhancement. We observe roughly an order-of-magnitude reduction in computational costs compared to conventional networks, with minimal reductions in model accuracy.