Being curious about the answers to questions: novelty search with learned attention

TL;DR

This paper explores using attentional neural networks to learn behavior characterizations that enhance curiosity-driven exploration and active inference, leading to more efficient discovery in exploration and guessing tasks.

Contribution

It introduces a novel method combining attention mechanisms with curiosity search to improve exploration and inference without reinforcement learning.

Findings

Attention-structured space encodes local contingencies effectively.

Greedy curiosity policy explores space quickly in 2D task.

Learned attention profile accelerates guessing in number game.

Abstract

We investigate the use of attentional neural network layers in order to learn a `behavior characterization' which can be used to drive novelty search and curiosity-based policies. The space is structured towards answering a particular distribution of questions, which are used in a supervised way to train the attentional neural network. We find that in a 2d exploration task, the structure of the space successfully encodes local sensory-motor contingencies such that even a greedy local `do the most novel action' policy with no reinforcement learning or evolution can explore the space quickly. We also apply this to a high/low number guessing game task, and find that guessing according to the learned attention profile performs active inference and can discover the correct number more quickly than an exact but passive approach.