Reinforcement Learning in Factored Action Spaces using Tensor Decompositions

TL;DR

This paper introduces a tensor decomposition-based approach for reinforcement learning in large, factored action spaces, particularly in multi-agent scenarios, to enable scalable and efficient learning.

Contribution

It extends previous work by applying tensor decompositions to approximate RL in factored action spaces, highlighting its potential in multi-agent systems.

Findings

Effective approximation of RL in large factored action spaces

Enhanced scalability in multi-agent reinforcement learning

Promotes tensor methods in the RL research community

Abstract

We present an extended abstract for the previously published work TESSERACT [Mahajan et al., 2021], which proposes a novel solution for Reinforcement Learning (RL) in large, factored action spaces using tensor decompositions. The goal of this abstract is twofold: (1) To garner greater interest amongst the tensor research community for creating methods and analysis for approximate RL, (2) To elucidate the generalised setting of factored action spaces where tensor decompositions can be used. We use cooperative multi-agent reinforcement learning scenario as the exemplary setting where the action space is naturally factored across agents and learning becomes intractable without resorting to approximation on the underlying hypothesis space for candidate solutions.