Augmented Segmentation and Visualization for Presentation Videos

TL;DR

This paper presents a method for segmenting, visualizing, and indexing presentation videos by combining audio speaker segmentation with key phrase extraction and visual dissimilarity-based video segmentation, enhanced by an interactive interface.

Contribution

It introduces a novel integrated approach for presentation video analysis that combines audio and visual data with interactive visualization tools.

Findings

Effective segmentation of audio by speaker and key phrases.

Video segmentation based on visual dissimilarities.

Prototype interface for multi-modal presentation navigation.

Abstract

We investigate methods of segmenting, visualizing, and indexing presentation videos by separately considering audio and visual data. The audio track is segmented by speaker, and augmented with key phrases which are extracted using an Automatic Speech Recognizer (ASR). The video track is segmented by visual dissimilarities and augmented by representative key frames. An interactive user interface combines a visual representation of audio, video, text, and key frames, and allows the user to navigate a presentation video. We also explore clustering and labeling of speaker data and present preliminary results.