Efficient Reinforcement Learning in Resource Allocation Problems Through Permutation Invariant Multi-task Learning

TL;DR

This paper introduces a neural network architecture and sampling strategy for multi-task reinforcement learning that leverages task invariance properties to significantly improve sample efficiency in resource allocation problems.

Contribution

The paper presents a novel multi-task learning approach exploiting invariance properties, with a theoretical performance bound and empirical validation on real-world tasks.

Findings

Enhanced sample efficiency in resource allocation RL tasks

Effective neural network architecture for invariant multi-task learning

Improved performance in financial portfolio optimization and federated learning

Abstract

One of the main challenges in real-world reinforcement learning is to learn successfully from limited training samples. We show that in certain settings, the available data can be dramatically increased through a form of multi-task learning, by exploiting an invariance property in the tasks. We provide a theoretical performance bound for the gain in sample efficiency under this setting. This motivates a new approach to multi-task learning, which involves the design of an appropriate neural network architecture and a prioritized task-sampling strategy. We demonstrate empirically the effectiveness of the proposed approach on two real-world sequential resource allocation tasks where this invariance property occurs: financial portfolio optimization and meta federated learning.