Using Inter-Sentence Diverse Beam Search to Reduce Redundancy in Visual Storytelling

TL;DR

This paper introduces an inter-sentence diverse beam search method to improve visual storytelling by reducing redundancy and enhancing story expressiveness, especially when images are similar, leading to more coherent and varied narratives.

Contribution

The paper proposes a novel inter-sentence diverse beam search technique that prevents repetitive sentences in visual storytelling, improving story quality over existing models.

Findings

Reduces sentence redundancy in visual storytelling.

Enhances story expressiveness with similar images.

Outperforms recent models in coherence and diversity.

Abstract

Visual storytelling includes two important parts: coherence between the story and images as well as the story structure. For image to text neural network models, similar images in the sequence would provide close information for story generator to obtain almost identical sentence. However, repeatedly narrating same objects or events will undermine a good story structure. In this paper, we proposed an inter-sentence diverse beam search to generate a more expressive story. Comparing to some recent models of visual storytelling task, which generate story without considering the generated sentence of the previous picture, our proposed method can avoid generating identical sentence even given a sequence of similar pictures.