Training slow silicon neurons to control extremely fast robots with spiking reinforcement learning

TL;DR

This paper demonstrates a neuromorphic spiking neural network system that learns to play air hockey in real-time, using hardware-software co-design and local learning rules to control fast-moving robots efficiently.

Contribution

It introduces a hardware-efficient, reinforcement learning approach with fixed connectivity and local e-prop learning for fast robotic control on neuromorphic hardware.

Findings

Successful real-time puck control in air hockey with few trials

Efficient event-driven learning on neuromorphic hardware

Bridging neuroscience-inspired hardware with robotic control

Abstract

Air hockey demands split-second decisions at high puck velocities, a challenge we address with a compact network of spiking neurons running on a mixed-signal analog/digital neuromorphic processor. By co-designing hardware and learning algorithms, we train the system to achieve successful puck interactions through reinforcement learning in a remarkably small number of trials. The network leverages fixed random connectivity to capture the task's temporal structure and adopts a local e-prop learning rule in the readout layer to exploit event-driven activity for fast and efficient learning. The result is real-time learning with a setup comprising a computer and the neuromorphic chip in-the-loop, enabling practical training of spiking neural networks for robotic autonomous systems. This work bridges neuroscience-inspired hardware with real-world robotic control, showing that brain-inspired approaches can tackle fast-paced interaction tasks while supporting always-on learning in intelligent machines.