Rewriting Video: Text-Driven Reauthoring of Video Footage

TL;DR

This paper explores text-driven video reauthoring by developing a generative reconstruction algorithm and an interactive tool, revealing new creative possibilities and challenges in editing videos through natural language prompts.

Contribution

It introduces a novel approach combining a generative reconstruction algorithm with an interactive tool for text-based video editing, providing empirical insights into its potential and limitations.

Findings

Identified a perceptual gap between human expectations and AI video reconstruction.

Discovered new use cases like virtual reshooting and aesthetic restyling.

Highlighted tensions around coherence, control, and creative alignment in text-driven editing.

Abstract

Video is a powerful medium for communication and storytelling, yet reauthoring existing footage remains challenging. Even simple edits often demand expertise, time, and careful planning, constraining how creators envision and shape their narratives. Recent advances in generative AI suggest a new paradigm: what if editing a video were as straightforward as rewriting text? To investigate this, we present a tech probe and a study on text-driven video reauthoring. Our approach involves two technical contributions: (1) a generative reconstruction algorithm that reverse-engineers video into an editable text prompt, and (2) an interactive probe, Rewrite Kit, that allows creators to manipulate these prompts. A technical evaluation of the algorithm reveals a critical human-AI perceptual gap. A probe study with 12 creators surfaced novel use cases such as virtual reshooting, synthetic continuity, and aesthetic restyling. It also highlighted key tensions around coherence, control, and creative alignment in this new paradigm. Our work contributes empirical insights into the opportunities and challenges of text-driven video reauthoring, offering design implications for future co-creative video tools.