Learning controllable dynamics through informative exploration

TL;DR

This paper introduces a method for exploring environments with unknown controllable dynamics by using predicted information gain to identify informative regions, enabling better learning of environment models through reinforcement learning.

Contribution

It proposes a novel exploration strategy based on predicted information gain, improving the learning of controllable dynamics without explicit models.

Findings

Outperforms myopic exploration methods in identifying informative regions.

Enables reliable estimation of environment dynamics.

Demonstrates effectiveness through comparative experiments.

Abstract

Environments with controllable dynamics are usually understood in terms of explicit models. However, such models are not always available, but may sometimes be learned by exploring an environment. In this work, we investigate using an information measure called "predicted information gain" to determine the most informative regions of an environment to explore next. Applying methods from reinforcement learning allows good suboptimal exploring policies to be found, and leads to reliable estimates of the underlying controllable dynamics. This approach is demonstrated by comparing with several myopic exploration approaches.