Angrier Birds: Bayesian reinforcement learning

TL;DR

This paper introduces RLSVI, a Bayesian reinforcement learning algorithm that enhances exploration efficiency in large state-action spaces, demonstrated through a simplified Angry Birds game.

Contribution

It presents RLSVI, a novel Bayesian approach to reinforcement learning that improves exploration efficiency over traditional methods like epsilon-greedy Q-learning.

Findings

RLSVI achieves faster learning in large state-action spaces.

Systematic exploration improves efficiency over epsilon-greedy methods.

Demonstrated effectiveness in a simplified Angry Birds environment.

Abstract

We train a reinforcement learner to play a simplified version of the game Angry Birds. The learner is provided with a game state in a manner similar to the output that could be produced by computer vision algorithms. We improve on the efficiency of regular {\epsilon}-greedy Q-Learning with linear function approximation through more systematic exploration in Randomized Least Squares Value Iteration (RLSVI), an algorithm that samples its policy from a posterior distribution on optimal policies. With larger state-action spaces, efficient exploration becomes increasingly important, as evidenced by the faster learning in RLSVI.