Diffusing Surrogate Dreams of Video Scenes to Predict Video Memorability

TL;DR

This paper introduces a novel approach to predicting video memorability by using surrogate dream images, emphasizing the importance of underlying concepts over visual details, and achieves state-of-the-art results.

Contribution

The study demonstrates that concepts extracted from surrogate dream images significantly improve video memorability prediction, surpassing previous methods.

Findings

State-of-the-art memorability prediction performance.

Conceptual features are crucial for memorability modeling.

Intrinsic memorability relates more to underlying concepts than visual details.

Abstract

As part of the MediaEval 2022 Predicting Video Memorability task we explore the relationship between visual memorability, the visual representation that characterises it, and the underlying concept portrayed by that visual representation. We achieve state-of-the-art memorability prediction performance with a model trained and tested exclusively on surrogate dream images, elevating concepts to the status of a cornerstone memorability feature, and finding strong evidence to suggest that the intrinsic memorability of visual content can be distilled to its underlying concept or meaning irrespective of its specific visual representational.