Making AI Drafts Count: A Quality Threshold in Audio Description Workflows

TL;DR

This paper explores how the quality of AI-generated drafts impacts the efficiency and effectiveness of audio description editing, emphasizing a minimum quality threshold for useful AI assistance.

Contribution

It introduces GenAD and RefineAD, demonstrating that high-quality AI drafts significantly improve audio description workflows and establish a content-dependent quality threshold.

Findings

GenAD drafts reduce completion time by over 50%

Higher-quality AI drafts lower cognitive load during editing

Content complexity influences the required AI draft quality

Abstract

Audio description (AD) narrates visual elements in video for blind and low-vision audiences. Recent work has shown that giving novice describers an AI-generated draft to start from helps produce higher-quality AD and lowers the barrier to entry. What remains an open question is how draft quality shapes the editing process. We investigate this through GenAD, an AD generation pipeline that incorporates accessibility guidelines and contextual video information, and RefineAD, an editing interface for human revisions. Human-AI contributions are measured across text, timing, and delivery. In a within-subjects study, we compared authoring from scratch against editing AI drafts of varying quality. GenAD drafts cut completion time by more than half and significantly reduced cognitive load. In contrast, baseline drafts generated from simple, unguided prompts offered only modest benefits, pointing to a minimum quality threshold for effectiveness. Qualitative findings suggest this threshold is content-dependent; as visual complexity increases, so does the quality needed from AI drafts. We propose this as a design principle: effective AI assistance should clear a quality threshold suited to the target content, rather than simply be present.