Learning To Explore With Predictive World Model Via Self-Supervised Learning

TL;DR

This paper introduces a self-supervised learning approach for autonomous agents to develop complex behaviors in environments by building an internal world model with cognitive elements, outperforming existing methods in Atari games.

Contribution

It presents a novel method that integrates cognitive elements into a predictive world model for intrinsically motivated learning without predefined rewards.

Findings

Outperforms state-of-the-art in many Atari games

Learns complex behaviors without manually designed rewards

Effective in environments with dense and sparse rewards

Abstract

Autonomous artificial agents must be able to learn behaviors in complex environments without humans to design tasks and rewards. Designing these functions for each environment is not feasible, thus, motivating the development of intrinsic reward functions. In this paper, we propose using several cognitive elements that have been neglected for a long time to build an internal world model for an intrinsically motivated agent. Our agent performs satisfactory iterations with the environment, learning complex behaviors without needing previously designed reward functions. We used 18 Atari games to evaluate what cognitive skills emerge in games that require reactive and deliberative behaviors. Our results show superior performance compared to the state-of-the-art in many test cases with dense and sparse rewards.