Reinforcement Learning, Bit by Bit

TL;DR

This paper explores principles of data efficiency in reinforcement learning, focusing on information acquisition and retention, and demonstrates simple agents that improve data efficiency through theoretical insights and computational results.

Contribution

It introduces a conceptual framework and regret analysis for understanding data-efficient reinforcement learning, along with simple agent designs illustrating these ideas.

Findings

Principled guidance on what information to seek and retain

Simple agents demonstrating improved data efficiency

Computational results validating the concepts

Abstract

Reinforcement learning agents have demonstrated remarkable achievements in simulated environments. Data efficiency poses an impediment to carrying this success over to real environments. The design of data-efficient agents calls for a deeper understanding of information acquisition and representation. We discuss concepts and regret analysis that together offer principled guidance. This line of thinking sheds light on questions of what information to seek, how to seek that information, and what information to retain. To illustrate concepts, we design simple agents that build on them and present computational results that highlight data efficiency.