Assisting News Media Editors with Cohesive Visual Storylines

TL;DR

This paper introduces a graph-based framework to assist news editors in creating cohesive visual storylines by predicting transition quality between images, demonstrating high accuracy and effectiveness in real-world news scenarios.

Contribution

It formalizes visual story transition and proposes four graph algorithms for cohesive storyline creation, leveraging a trained ensemble estimator for transition quality prediction.

Findings

Visual transition prediction accuracy exceeds 90%.

Graph methods produce storylines with 88-96% quality.

User study confirms effectiveness in real news stories.

Abstract

Creating a cohesive, high-quality, relevant, media story is a challenge that news media editors face on a daily basis. This challenge is aggravated by the flood of highly relevant information that is constantly pouring onto the newsroom. To assist news media editors in this daunting task, this paper proposes a framework to organize news content into cohesive, high-quality, relevant visual storylines. First, we formalize, in a nonsubjective manner, the concept of visual story transition. Leveraging it, we propose four graph-based methods of storyline creation, aiming for global story cohesiveness. These were created and implemented to take full advantage of existing graph algorithms, ensuring their correctness and good computational performance. They leverage a strong ensemble-based estimator which was trained to predict story transition quality based on both the semantic and visual features present in the pair of images under scrutiny. A user study covered a total of 28 curated stories about sports and cultural events. Experiments showed that (i) visual transitions in storylines can be learned with a quality above 90%, and (ii) the proposed graph methods can produce cohesive storylines with quality in the range of 88% to 96%.