Model-Size Reduction for Reservoir Computing by Concatenating Internal States Through Time

TL;DR

This paper introduces methods to significantly reduce the size of reservoirs in reservoir computing by concatenating internal states, enabling more efficient edge computing without sacrificing much accuracy.

Contribution

The study proposes novel reservoir size reduction techniques that utilize past or drifting states, validated through information capacity and time-series prediction tasks.

Findings

Reservoir size reduced to one-tenth with minimal error increase.

Methods applicable across various reservoir network structures.

Potential for improved energy efficiency in hardware implementations.

Abstract

Reservoir computing (RC) is a machine learning algorithm that can learn complex time series from data very rapidly based on the use of high-dimensional dynamical systems, such as random networks of neurons, called "reservoirs." To implement RC in edge computing, it is highly important to reduce the amount of computational resources that RC requires. In this study, we propose methods that reduce the size of the reservoir by inputting the past or drifting states of the reservoir to the output layer at the current time step. These proposed methods are analyzed based on information processing capacity, which is a performance measure of RC proposed by Dambre et al. (2012). In addition, we evaluate the effectiveness of the proposed methods on time-series prediction tasks: the generalized Henon-map and NARMA. On these tasks, we found that the proposed methods were able to reduce the size of the reservoir up to one tenth without a substantial increase in regression error. Because the applications of the proposed methods are not limited to a specific network structure of the reservoir, the proposed methods could further improve the energy efficiency of RC-based systems, such as FPGAs and photonic systems.