What Makes Natural Scene Memorable?

TL;DR

This paper investigates the factors influencing natural scene memorability by creating a large database, analyzing features, and proposing a deep learning model that leverages scene category information.

Contribution

It introduces LNSIM, a large-scale natural scene image database, and develops DeepNSM, a neural network model utilizing scene category for memorability prediction.

Findings

High-level scene category correlates strongly with memorability.

DeepNSM outperforms existing models in predicting scene memorability.

Scene features at various levels influence memorability differently.

Abstract

Recent studies on image memorability have shed light on the visual features that make generic images, object images or face photographs memorable. However, a clear understanding and reliable estimation of natural scene memorability remain elusive. In this paper, we provide an attempt to answer: "what exactly makes natural scene memorable". Specifically, we first build LNSIM, a large-scale natural scene image memorability database (containing 2,632 images and memorability annotations). Then, we mine our database to investigate how low-, middle- and high-level handcrafted features affect the memorability of natural scene. In particular, we find that high-level feature of scene category is rather correlated with natural scene memorability. Thus, we propose a deep neural network based natural scene memorability (DeepNSM) predictor, which takes advantage of scene category. Finally, the experimental results validate the effectiveness of DeepNSM.