Analysis and Visualization of Index Words from Audio Transcripts of Instructional Videos

TL;DR

This paper presents techniques for extracting, analyzing, and visualizing key textual content from low-quality instructional videos using speech recognition and filtering, enabling better understanding and indexing of course materials.

Contribution

It introduces novel visualization tools and methods for analyzing transcripts from low-quality videos, including key phrase extraction and semantic clustering, with demonstrated effectiveness on multiple courses.

Findings

Up to 98 key terms extracted per transcript

Textbook match accuracy exceeds 70%

Effective visualization of course structure and topics

Abstract

We introduce new techniques for extracting, analyzing, and visualizing textual contents from instructional videos of low production quality. Using Automatic Speech Recognition, approximate transcripts (H75% Word Error Rate) are obtained from the originally highly compressed videos of university courses, each comprising between 10 to 30 lectures. Text material in the form of books or papers that accompany the course are then used to filter meaningful phrases from the seemingly incoherent transcripts. The resulting index into the transcripts is tied together and visualized in 3 experimental graphs that help in understanding the overall course structure and provide a tool for localizing certain topics for indexing. We specifically discuss a Transcript Index Map, which graphically lays out key phrases for a course, a Textbook Chapter to Transcript Match, and finally a Lecture Transcript Similarity graph, which clusters semantically similar lectures. We test our methods and tools on 7 full courses with 230 hours of video and 273 transcripts. We are able to extract up to 98 unique key terms for a given transcript and up to 347 unique key terms for an entire course. The accuracy of the Textbook Chapter to Transcript Match exceeds 70% on average. The methods used can be applied to genres of video in which there are recurrent thematic words (news, sports, meetings,...)