Hard Attention Control By Mutual Information Maximization

TL;DR

This paper introduces a method for controlling a hard attention window in artificial agents by maximizing mutual information, enabling effective partial observation and state prediction in partially observable environments.

Contribution

It presents a novel approach combining mutual information maximization with a dynamic memory architecture for attention control in partially observable tasks.

Findings

Effective full state prediction from partial observations.

Successful control of attention in reinforcement learning tasks.

Internal world model aids in decision making.

Abstract

Biological agents have adopted the principle of attention to limit the rate of incoming information from the environment. One question that arises is if an artificial agent has access to only a limited view of its surroundings, how can it control its attention to effectively solve tasks? We propose an approach for learning how to control a hard attention window by maximizing the mutual information between the environment state and the attention location at each step. The agent employs an internal world model to make predictions about its state and focuses attention towards where the predictions may be wrong. Attention is trained jointly with a dynamic memory architecture that stores partial observations and keeps track of the unobserved state. We demonstrate that our approach is effective in predicting the full state from a sequence of partial observations. We also show that the agent's internal representation of the surroundings, a live mental map, can be used for control in two partially observable reinforcement learning tasks. Videos of the trained agent can be found at https://sites.google.com/view/hard-attention-control.