Intrinsic Motivation in Model-based Reinforcement Learning: A Brief Review

TL;DR

This paper reviews methods of intrinsic motivation in model-based reinforcement learning, categorizing approaches based on how they utilize world models to enhance agent autonomy and learning efficiency.

Contribution

It introduces a unified framework categorizing intrinsic motivation methods in model-based RL and discusses potential directions for future research.

Findings

Categorizes intrinsic motivation methods into three types: intrinsic reward, exploration policy, and motivated goals.

Proposes a unified framework for agents using world models and intrinsic motivation.

Highlights potential for developing new techniques in intrinsic motivation for RL.

Abstract

The reinforcement learning research area contains a wide range of methods for solving the problems of intelligent agent control. Despite the progress that has been made, the task of creating a highly autonomous agent is still a significant challenge. One potential solution to this problem is intrinsic motivation, a concept derived from developmental psychology. This review considers the existing methods for determining intrinsic motivation based on the world model obtained by the agent. We propose a systematic approach to current research in this field, which consists of three categories of methods, distinguished by the way they utilize a world model in the agent's components: complementary intrinsic reward, exploration policy, and intrinsically motivated goals. The proposed unified framework describes the architecture of agents using a world model and intrinsic motivation to improve learning. The potential for developing new techniques in this area of research is also examined.