Reinforcement Learning in Hyperbolic Spaces: Models and Experiments

TL;DR

This paper explores reinforcement learning in hyperbolic spaces, introducing models and algorithms for environments where action spaces are endowed with hyperbolic metrics, broadening RL applications.

Contribution

It formalizes RL problems in hyperbolic spaces and develops statistical and dynamical models along with algorithms for such settings.

Findings

RL in hyperbolic spaces can model diverse exploration problems

New algorithms effectively operate in hyperbolic action spaces

Framework unifies various RL exploration scenarios

Abstract

We examine five setups where an agent (or two agents) seeks to explore unknown environment without any prior information. Although seemingly very different, all of them can be formalized as Reinforcement Learning (RL) problems in hyperbolic spaces. More precisely, it is natural to endow the action spaces with the hyperbolic metric. We introduce statistical and dynamical models necessary for addressing problems of this kind and implement algorithms based on this framework. Throughout the paper we view RL through the lens of the black-box optimization.