On the Unreasonable Efficiency of State Space Clustering in Personalization Tasks

TL;DR

This paper demonstrates that simple state space clustering using k-means significantly accelerates reinforcement learning in personalization tasks without compromising performance.

Contribution

It introduces a straightforward clustering-based RL method that improves learning speed in complex personalization environments.

Findings

Clustering accelerates RL training.

The method maintains high performance levels.

Simple algorithms suffice for effective personalization RL.

Abstract

In this effort we consider a reinforcement learning (RL) technique for solving personalization tasks with complex reward signals. In particular, our approach is based on state space clustering with the use of a simplistic $k$-means algorithm as well as conventional choices of the network architectures and optimization algorithms. Numerical examples demonstrate the efficiency of different RL procedures and are used to illustrate that this technique accelerates the agent's ability to learn and does not restrict the agent's performance.