More than a Moment: Towards Coherent Sequences of Audio Descriptions

TL;DR

This paper introduces CoherentAD, a training-free method for generating coherent audio description sequences that improve narrative flow and reduce redundancy, evaluated with a new sequence-level metric called StoryRecall.

Contribution

The paper presents a novel training-free approach for generating coherent audio description sequences and introduces the StoryRecall metric for holistic evaluation.

Findings

CoherentAD outperforms prior independent generation methods.

The sequence-level metric StoryRecall effectively measures narrative coherence.

Enhanced narrative understanding demonstrated in experimental results.

Abstract

Audio Descriptions (ADs) convey essential on-screen information, allowing visually impaired audiences to follow videos. To be effective, ADs must form a coherent sequence that helps listeners to visualise the unfolding scene, rather than describing isolated moments. However, most automatic methods generate each AD independently, often resulting in repetitive, incoherent descriptions. To address this, we propose a training-free method, CoherentAD, that first generates multiple candidate descriptions for each AD time interval, and then performs auto-regressive selection across the sequence to form a coherent and informative narrative. To evaluate AD sequences holistically, we introduce a sequence-level metric, StoryRecall, which measures how well the predicted ADs convey the ground truth narrative, alongside repetition metrics that capture the redundancy across consecutive AD outputs. Our method produces coherent AD sequences with enhanced narrative understanding, outperforming prior approaches that rely on independent generations.