Segmentation, Indexing, and Visualization of Extended Instructional Videos

TL;DR

This paper introduces a new method for segmenting, indexing, and visualizing extended instructional videos, enabling efficient navigation through semantic content with high accuracy.

Contribution

It presents a novel segmentation algorithm and a user interface for visualizing instructional videos, with high classification accuracy and user-centered design.

Findings

Segmentation accuracy exceeds 96% on 17 instructional videos.

Clustering key frames based on media type and content is efficient and effective.

The user interface allows quick access to related video topics.

Abstract

We present a new method for segmenting, and a new user interface for indexing and visualizing, the semantic content of extended instructional videos. Given a series of key frames from the video, we generate a condensed view of the data by clustering frames according to media type and visual similarities. Using various visual filters, key frames are first assigned a media type (board, class, computer, illustration, podium, and sheet). Key frames of media type board and sheet are then clustered based on contents via an algorithm with near-linear cost. A novel user interface, the result of two user studies, displays related topics using icons linked topologically, allowing users to quickly locate semantically related portions of the video. We analyze the accuracy of the segmentation tool on 17 instructional videos, each of which is from 75 to 150 minutes in duration (a total of 40 hours); the classification accuracy exceeds 96%.