Pre-trainable Reservoir Computing with Recursive Neural Gas

TL;DR

This paper introduces an improved model of Recursive Neural Gas (RNG) for reservoir computing, compares it with Echo State Networks (ESN), and demonstrates that RNG can outperform ESN on certain benchmark datasets.

Contribution

The paper presents an accurate, extended RNG model and provides the first comparative experimental results against ESN on standard benchmarks.

Findings

RNG-based reservoirs can outperform ESN in specific scenarios

Extended RNG models improve performance over previous versions

Experimental results validate RNG's potential in reservoir computing

Abstract

Echo State Networks (ESN) are a class of Recurrent Neural Networks (RNN) that has gained substantial popularity due to their effectiveness, ease of use and potential for compact hardware implementation. An ESN contains the three network layers input, reservoir and readout where the reservoir is the truly recurrent network. The input and reservoir layers of an ESN are initialized at random and never trained afterwards and the training of the ESN is applied to the readout layer only. The alternative of Recursive Neural Gas (RNG) is one of the many proposals of fully-trainable reservoirs that can be found in the literature. Although some improvements in performance have been reported with RNG, to the best of authors' knowledge, no experimental comparative results are known with benchmarks for which ESN is known to yield excellent results. This work describes an accurate model of RNG together with some extensions to the models presented in the literature and shows comparative results on three well-known and accepted datasets. The experimental results obtained show that, under specific circumstances, RNG-based reservoirs can achieve better performance.