Randomness and isometries in echo state networks and compressed sensing

TL;DR

This paper investigates the parallels between echo state networks and compressed sensing, proposing a new metric inspired by restricted isometry properties to improve echo state network performance.

Contribution

It introduces a novel restricted isometry-inspired metric for echo state networks and demonstrates its effectiveness through theoretical analysis and experiments.

Findings

Random matrices effective for both models

Improved classification accuracy with the new metric

Theoretical and experimental validation of methods

Abstract

Although largely different concepts, echo state networks and compressed sensing models both rely on collections of random weights; as the reservoir dynamics for echo state networks, and the sensing coefficients in compressed sensing. Several methods for generating the random matrices and metrics to indicate desirable performance are well-studied in compressed sensing, but less so for echo state networks. This work explores any overlap in these compressed sensing methods and metrics for application to echo state networks. Several methods for generating the random reservoir weights are considered, and a new metric, inspired by the restricted isometry property for compressed sensing, is proposed for echo state networks. The methods and metrics are investigated theoretically and experimentally, with results suggesting that the same types of random matrices work well for both echo state network and compressed sensing scenarios, and that echo state network classification accuracy is improved when the proposed restricted isometry-like constants are close to 1.