Learning to predict where to look in interactive environments using deep recurrent q-learning

TL;DR

This paper introduces a reinforcement learning approach with a soft attention mechanism to predict where to look in interactive environments, outperforming traditional bottom-up saliency models in complex tasks like Atari games.

Contribution

It combines deep Q-learning with a soft attention mechanism to improve prediction of task-relevant fixation points in interactive environments.

Findings

Outperforms bottom-up saliency models in predicting fixation locations.

Effective in complex interactive tasks like Atari games.

Demonstrates improved focus on relevant visual input.

Abstract

Bottom-Up (BU) saliency models do not perform well in complex interactive environments where humans are actively engaged in tasks (e.g., sandwich making and playing the video games). In this paper, we leverage Reinforcement Learning (RL) to highlight task-relevant locations of input frames. We propose a soft attention mechanism combined with the Deep Q-Network (DQN) model to teach an RL agent how to play a game and where to look by focusing on the most pertinent parts of its visual input. Our evaluations on several Atari 2600 games show that the soft attention based model could predict fixation locations significantly better than bottom-up models such as Itti-Kochs saliency and Graph-Based Visual Saliency (GBVS) models.