Script-to-Slide Grounding: Grounding Script Sentences to Slide Objects for Automatic Instructional Video Generation
Rena Suzuki, Masato Kikuchi, Tadachika Ozono

TL;DR
This paper introduces Script-to-Slide Grounding (S2SG), a formalized task for automatically linking script sentences to slide objects, enabling automation of instructional video creation with high accuracy using LLMs.
Contribution
It formalizes the implicit process of slide-based video editing into a computable task and proposes a method leveraging large language models for effective grounding.
Findings
Achieved high grounding performance with F1-score of 0.924
Formalized the grounding task to facilitate automation
Demonstrated the effectiveness of LLMs in script-to-slide grounding
Abstract
While slide-based videos augmented with visual effects are widely utilized in education and research presentations, the video editing process -- particularly applying visual effects to ground spoken content to slide objects -- remains highly labor-intensive. This study aims to develop a system that automatically generates such instructional videos from slides and corresponding scripts. As a foundational step, this paper proposes and formulates Script-to-Slide Grounding (S2SG), defined as the task of grounding script sentences to their corresponding slide objects. Furthermore, as an initial step, we propose ``Text-S2SG,'' a method that utilizes a large language model (LLM) to perform this grounding task for text objects. Our experiments demonstrate that the proposed method achieves high performance (F1-score: 0.924). The contribution of this work is the formalization of a previously…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsMultimodal Machine Learning Applications · Video Analysis and Summarization · Human Motion and Animation
