Attend to what I say: Highlighting relevant content on slides

TL;DR

This paper presents a method to automatically identify and highlight relevant slide regions during presentations by analyzing spoken narration to improve viewer comprehension and synchronization.

Contribution

It introduces a novel approach that matches spoken content with slide elements to highlight relevant regions, enhancing understanding in content-rich multimedia presentations.

Findings

Effective identification of relevant slide regions based on speech analysis

Improved synchronization between spoken narration and slide content

Assessment of different solutions with success and failure cases

Abstract

Imagine sitting in a presentation, trying to follow the speaker while simultaneously scanning the slides for relevant information. While the entire slide is visible, identifying the relevant regions can be challenging. As you focus on one part of the slide, the speaker moves on to a new sentence, leaving you scrambling to catch up visually. This constant back-and-forth creates a disconnect between what is being said and the most important visual elements, making it hard to absorb key details, especially in fast-paced or content-heavy presentations such as conference talks. This requires an understanding of slides, including text, graphics, and layout. We introduce a method that automatically identifies and highlights the most relevant slide regions based on the speaker's narrative. By analyzing spoken content and matching it with textual or graphical elements in the slides, our approach ensures better synchronization between what listeners hear and what they need to attend to. We explore different ways of solving this problem and assess their success and failure cases. Analyzing multimedia documents is emerging as a key requirement for seamless understanding of content-rich videos, such as educational videos and conference talks, by reducing cognitive strain and improving comprehension. Code and dataset are available at: https://github.com/meghamariamkm2002/Slide_Highlight