The Sample-Complexity of General Reinforcement Learning

TL;DR

This paper introduces a new reinforcement learning algorithm with near-optimal sample complexity for finite environment classes and explores the importance of compactness for infinite classes.

Contribution

It provides a near-optimal algorithm for finite classes and establishes the role of compactness in infinite classes for sample complexity bounds.

Findings

Near-optimal sample complexity for finite classes

Compactness as a key criterion for infinite classes

Matching lower bound for finite environment classes

Abstract

We present a new algorithm for general reinforcement learning where the true environment is known to belong to a finite class of N arbitrary models. The algorithm is shown to be near-optimal for all but O(N log^2 N) time-steps with high probability. Infinite classes are also considered where we show that compactness is a key criterion for determining the existence of uniform sample-complexity bounds. A matching lower bound is given for the finite case.