AMNet: Memorability Estimation with Attention

TL;DR

This paper introduces AMNet, a deep neural network with attention for estimating image memorability, demonstrating superior performance over previous models and aligning closely with human memorability judgments.

Contribution

The paper proposes a novel attention-based deep learning model for memorability estimation, exploring transfer learning and demonstrating state-of-the-art results.

Findings

AMNet outperforms existing models on SUN and LaMem datasets.

The attention mechanism improves memorability prediction accuracy.

The model closely matches human memorability consistency.

Abstract

In this paper we present the design and evaluation of an end-to-end trainable, deep neural network with a visual attention mechanism for memorability estimation in still images. We analyze the suitability of transfer learning of deep models from image classification to the memorability task. Further on we study the impact of the attention mechanism on the memorability estimation and evaluate our network on the SUN Memorability and the LaMem datasets. Our network outperforms the existing state of the art models on both datasets in terms of the Spearman's rank correlation as well as the mean squared error, closely matching human consistency.