Generating Memorable Images Based on Human Visual Memory Schemas

TL;DR

This paper introduces a GAN-based method that incorporates a human memory-inspired 2D memorability measure to generate images that are either memorable or non-memorable, evaluated through a VMS model.

Contribution

It proposes a novel approach to control image memorability in GANs using a Visual Memory Schemas map as an auxiliary loss, bridging human memory models and image generation.

Findings

Generated images show significant differences in memorability scores.

Memorability constraints influence the perceived realness of images.

The VMS-based approach effectively modulates image memorability.

Abstract

This research study proposes using Generative Adversarial Networks (GAN) that incorporate a two-dimensional measure of human memorability to generate memorable or non-memorable images of scenes. The memorability of the generated images is evaluated by modelling Visual Memory Schemas (VMS), which correspond to mental representations that human observers use to encode an image into memory. The VMS model is based upon the results of memory experiments conducted on human observers, and provides a 2D map of memorability. We impose a memorability constraint upon the latent space of a GAN by employing a VMS map prediction model as an auxiliary loss. We assess the difference in memorability between images generated to be memorable or non-memorable through an independent computational measure of memorability, and additionally assess the effect of memorability on the realness of the generated images.