The many faces of optimism - Extended version

TL;DR

This paper introduces a fast, simple algorithm for reinforcement learning that effectively balances exploration and exploitation, demonstrating near-optimal performance and robustness through theoretical analysis and experiments.

Contribution

It presents a novel, integrated approach that combines optimism and model building to achieve efficient, near-optimal policies in polynomial time.

Findings

Algorithm finds near-optimal policies efficiently

Demonstrates robustness and efficiency in experiments

Provides theoretical guarantees of polynomial-time convergence

Abstract

The exploration-exploitation dilemma has been an intriguing and unsolved problem within the framework of reinforcement learning. "Optimism in the face of uncertainty" and model building play central roles in advanced exploration methods. Here, we integrate several concepts and obtain a fast and simple algorithm. We show that the proposed algorithm finds a near-optimal policy in polynomial time, and give experimental evidence that it is robust and efficient compared to its ascendants.