# Reinforcement Learning with Low-Complexity Liquid State Machines

## TL;DR

This paper introduces a low-complexity reinforcement learning method using sparse, randomly connected spiking neural networks that efficiently learn complex tasks with minimal trainable parameters, comparable to human performance.

## Contribution

It presents a novel approach combining simple random spiking networks with Q-learning, offering an energy-efficient alternative to deep reinforcement learning.

## Key findings

- Achieves human-level performance on Atari games
- Uses significantly fewer trainable parameters
- Provides a computationally efficient, spike-based learning method

## Abstract

We propose reinforcement learning on simple networks consisting of random connections of spiking neurons (both recurrent and feed-forward) that can learn complex tasks with very little trainable parameters. Such sparse and randomly interconnected recurrent spiking networks exhibit highly non-linear dynamics that transform the inputs into rich high-dimensional representations based on past context. The random input representations can be efficiently interpreted by an output (or readout) layer with trainable parameters. Systematic initialization of the random connections and training of the readout layer using Q-learning algorithm enable such small random spiking networks to learn optimally and achieve the same learning efficiency as humans on complex reinforcement learning tasks like Atari games. The spike-based approach using small random recurrent networks provides a computationally efficient alternative to state-of-the-art deep reinforcement learning networks with several layers of trainable parameters. The low-complexity spiking networks can lead to improved energy efficiency in event-driven neuromorphic hardware for complex reinforcement learning tasks.