Fast Reinforcement Learning with Incremental Gaussian Mixture Models

TL;DR

This paper introduces a data-efficient reinforcement learning algorithm using an incremental Gaussian mixture model that learns from minimal data and offers advantages over traditional neural network approaches.

Contribution

The paper presents the Incremental Gaussian Mixture Network (IGMN), a novel online algorithm for reinforcement learning that efficiently approximates functions in continuous spaces with minimal data.

Findings

IGMN enables learning from very few environment interactions.

It offers advantages over neural networks trained by gradient descent.

The algorithm is concise and data-efficient.

Abstract

This work presents a novel algorithm that integrates a data-efficient function approximator with reinforcement learning in continuous state spaces. An online and incremental algorithm capable of learning from a single pass through data, called Incremental Gaussian Mixture Network (IGMN), was employed as a sample-efficient function approximator for the joint state and Q-values space, all in a single model, resulting in a concise and data-efficient algorithm, i.e., a reinforcement learning algorithm that learns from very few interactions with the environment. Results are analyzed to explain the properties of the obtained algorithm, and it is observed that the use of the IGMN function approximator brings some important advantages to reinforcement learning in relation to conventional neural networks trained by gradient descent methods.