Asynchrony Increases Efficiency: Time Encoding of Videos and Low-Rank Signals

TL;DR

This paper demonstrates that asynchronous event-based sensing, modeled by time encoding machines, enhances efficiency and resolution in signal and video reconstruction by leveraging low-rank structures and asynchronous firing times.

Contribution

It introduces a theoretical framework for time encoding of low-rank signals, applying it to videos, and shows how asynchrony improves reconstruction quality over traditional sampling methods.

Findings

Asynchronous firing improves signal reconstruction quality.

Time encoding of low-rank signals outperforms standard sampling.

High spatial density of encoding machines enhances efficiency.

Abstract

In event-based sensing, many sensors independently and asynchronously emit events when there is a change in their input. Event-based sensing can present significant improvements in power efficiency when compared to traditional sampling, because (1) the output is a stream of events where the important information lies in the timing of the events, and (2) the sensor can easily be controlled to output information only when interesting activity occurs at the input. Moreover, event-based sampling can often provide better resolution than standard uniform sampling. Not only does this occur because individual event-based sensors have higher temporal resolution, it also occurs because the asynchrony of events allows for less redundant and more informative encoding. We would like to explain how such curious results come about. To do so, we use ideal time encoding machines as a proxy for event-based sensors. We explore time encoding of signals with low rank structure, and apply the resulting theory to video. We then see how the asynchronous firing times of the time encoding machines allow for better reconstruction than in the standard sampling case, if we have a high spatial density of time encoding machines that fire less frequently.