Optimal Neuron Selection: NK Echo State Networks for Reinforcement Learning

TL;DR

This paper presents the NK Echo State Network, a neuron selection method for reinforcement learning that optimizes neuron activation states without weight adjustments, achieving rapid learning and strong generalization.

Contribution

It introduces the NK Echo State Network, a novel neuron selection approach that reduces learning to an NK Landscape optimization, enabling efficient global solutions.

Findings

Learns rapidly in reinforcement learning tasks

Achieves strong generalization performance

Optimizes neuron states without weight updates

Abstract

This paper introduces the NK Echo State Network. The problem of learning in the NK Echo State Network is reduced to the problem of optimizing a special form of a Spin Glass Problem known as an NK Landscape. No weight adjustment is used; all learning is accomplished by spinning up (turning on) or spinning down (turning off) neurons in order to find a combination of neurons that work together to achieve the desired computation. For special types of NK Landscapes, an exact global solution can be obtained in polynomial time using dynamic programming. The NK Echo State Network is applied to a reinforcement learning problem requiring a recurrent network: balancing two poles on a cart given no velocity information. Empirical results shows that the NK Echo State Network learns very rapidly and yields very good generalization.