CERiL: Continuous Event-based Reinforcement Learning

TL;DR

This paper introduces CERiL, a novel continuous time reinforcement learning framework utilizing event cameras and specialized neural network layers to improve reactivity and performance over traditional methods.

Contribution

The paper formalizes continuous time RL with event streams and develops CERiL, a new algorithm that directly processes event data, outperforming traditional RL networks and SNN baselines.

Findings

CERiL outperforms traditional RL networks on several tasks.

Event streams provide advantages over RGB images in RL.

CERiL succeeds in tasks where traditional methods fail.

Abstract

This paper explores the potential of event cameras to enable continuous time reinforcement learning. We formalise this problem where a continuous stream of unsynchronised observations is used to produce a corresponding stream of output actions for the environment. This lack of synchronisation enables greatly enhanced reactivity. We present a method to train on event streams derived from standard RL environments, thereby solving the proposed continuous time RL problem. The CERiL algorithm uses specialised network layers which operate directly on an event stream, rather than aggregating events into quantised image frames. We show the advantages of event streams over less-frequent RGB images. The proposed system outperforms networks typically used in RL, even succeeding at tasks which cannot be solved traditionally. We also demonstrate the value of our CERiL approach over a standard SNN baseline using event streams.