Modeling Visual Memorability Assessment with Autoencoders Reveals Characteristics of Memorable Images

TL;DR

This paper introduces a deep learning autoencoder approach to model and analyze what makes images memorable, revealing key visual features and their relationship with memorability scores.

Contribution

It presents a novel autoencoder-based framework trained on VGG16 to predict and interpret image memorability, linking latent representations with human memory characteristics.

Findings

Reconstruction error correlates with memorability scores.

Latent space features are significantly distinctive for memorable images.

Identified visual features contribute to image memorability.

Abstract

Image memorability refers to the phenomenon where certain images are more likely to be remembered than others. It is a quantifiable and intrinsic image attribute, defined as the likelihood of an image being remembered upon a single exposure. Despite advances in understanding human visual perception and memory, it is unclear what features contribute to an image's memorability. To address this question, we propose a deep learning-based computational modeling approach. We employ an autoencoder-based approach built on VGG16 convolutional neural networks (CNNs) to learn latent representations of images. The model is trained in a single-epoch setting, mirroring human memory experiments that assess recall after a single exposure. We examine the relationship between autoencoder reconstruction error and memorability, analyze the distinctiveness of latent space representations, and develop a multi-layer perceptron (MLP) model for memorability prediction. Additionally, we perform interpretability analysis using Integrated Gradients (IG) to identify the key visual characteristics that contribute to memorability. Our results demonstrate a significant correlation between the images' memorability score and the autoencoder's reconstruction error, as well as the robust predictive performance of its latent representations. Distinctiveness in these representations correlated significantly with memorability. Additionally, certain visual characteristics were identified as features contributing to image memorability in our model. These findings suggest that autoencoder-based representations capture fundamental aspects of image memorability, providing new insights into the computational modeling of human visual memory.