Toward Real-Time Decentralized Reinforcement Learning using Finite Support Basis Functions

TL;DR

This paper introduces a decentralized RL approach with finite support basis functions to enable real-time complex behaviors on resource-limited robots, significantly reducing computation and memory needs.

Contribution

It proposes using finite support basis functions with decentralized RL to address high-dimensionality and computational constraints in real-time robotic applications.

Findings

Achieves up to 99.94% reduction in execution time

Reduces memory consumption by 98.82%

Maintains performance comparable to classical methods

Abstract

This paper addresses the design and implementation of complex Reinforcement Learning (RL) behaviors where multi-dimensional action spaces are involved, as well as the need to execute the behaviors in real-time using robotic platforms with limited computational resources and training times. For this purpose, we propose the use of decentralized RL, in combination with finite support basis functions as alternatives to Gaussian RBF, in order to alleviate the effects of the curse of dimensionality on the action and state spaces respectively, and to reduce the computation time. As testbed, a RL based controller for the in-walk kick in NAO robots, a challenging and critical problem for soccer robotics, is used. The reported experiments show empirically that our solution saves up to 99.94% of execution time and 98.82% of memory consumption during execution, without diminishing performance compared to classical approaches.