Agent Spaces
John C. Raisbeck, Matthew W. Allen, Hakho Lee

TL;DR
This paper redefines exploration in Reinforcement Learning as modifications to the agent itself, enabling its application to infinite and non-dynamic problems by introducing a novel agent space topology.
Contribution
It introduces a new exploration framework based on agent modifications and develops a topology on agent spaces, broadening RL applicability beyond finite, dynamic settings.
Findings
Defines exploration as agent modifications applicable to infinite problems.
Introduces a topology on agent space based on a collection of agent distances.
Shows that key RL structures are well-behaved under this topology.
Abstract
Exploration is one of the most important tasks in Reinforcement Learning, but it is not well-defined beyond finite problems in the Dynamic Programming paradigm (see Subsection 2.4). We provide a reinterpretation of exploration which can be applied to any online learning method. We come to this definition by approaching exploration from a new direction. After finding that concepts of exploration created to solve simple Markov decision processes with Dynamic Programming are no longer broadly applicable, we reexamine exploration. Instead of extending the ends of dynamic exploration procedures, we extend their means. That is, rather than repeatedly sampling every state-action pair possible in a process, we define the act of modifying an agent to itself be explorative. The resulting definition of exploration can be applied in infinite problems and non-dynamic learning methods, which the…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsReinforcement Learning in Robotics · Game Theory and Applications
