State-free Reinforcement Learning

TL;DR

This paper introduces a state-free reinforcement learning algorithm that operates without prior knowledge of the environment's state space, aiming for hyper-parameter free RL.

Contribution

It presents the first algorithm for state-free RL with regret bounds independent of the entire state space, advancing towards parameter-free RL.

Findings

Regret depends only on the reachable state set, not the entire state space.

Algorithm requires no prior state information.

Progress towards hyper-parameter free reinforcement learning.

Abstract

In this work, we study the \textit{state-free RL} problem, where the algorithm does not have the states information before interacting with the environment. Specifically, denote the reachable state set by ${S}^\Pi := \{ s|\max_{\pi\in \Pi}q^{P, \pi}(s)>0 \}$, we design an algorithm which requires no information on the state space $S$ while having a regret that is completely independent of ${S}$ and only depend on ${S}^\Pi$. We view this as a concrete first step towards \textit{parameter-free RL}, with the goal of designing RL algorithms that require no hyper-parameter tuning.