Provably Efficient Multi-Task Reinforcement Learning with Model Transfer

TL;DR

This paper introduces a model transfer-based algorithm for multi-task reinforcement learning in tabular MDPs, providing theoretical bounds that characterize the problem's complexity and improve collective performance through inter-player information sharing.

Contribution

It presents a novel algorithm for multi-task RL with theoretical analysis, including upper and lower bounds, demonstrating the benefits of model transfer and inter-player collaboration.

Findings

The algorithm achieves near-optimal performance bounds.

Inter-player information sharing improves learning efficiency.

Theoretical bounds characterize the problem's intrinsic complexity.

Abstract

We study multi-task reinforcement learning (RL) in tabular episodic Markov decision processes (MDPs). We formulate a heterogeneous multi-player RL problem, in which a group of players concurrently face similar but not necessarily identical MDPs, with a goal of improving their collective performance through inter-player information sharing. We design and analyze an algorithm based on the idea of model transfer, and provide gap-dependent and gap-independent upper and lower bounds that characterize the intrinsic complexity of the problem.