Multicopy Reinforcement Learning Agents

TL;DR

This paper introduces a multicopy reinforcement learning approach where multiple identical agents collaborate to improve performance in noisy or challenging environments, with a new algorithm that efficiently balances the benefits and costs of additional copies.

Contribution

It proposes a novel learning algorithm tailored for multicopy agents that leverages value function structure to optimize the number of copies for better performance.

Findings

Multicopy agents outperform single agents in noisy environments.

The proposed algorithm efficiently learns the optimal number of copies.

Results demonstrate improved task success rates with multicopy strategies.

Abstract

This paper examines a novel type of multi-agent problem, in which an agent makes multiple identical copies of itself in order to achieve a single agent task better or more efficiently. This strategy improves performance if the environment is noisy and the task is sometimes unachievable by a single agent copy. We propose a learning algorithm for this multicopy problem which takes advantage of the structure of the value function to efficiently learn how to balance the advantages and costs of adding additional copies.