Classification of Important Segments in Educational Videos using Multimodal Features

TL;DR

This paper introduces a multimodal neural approach to automatically identify and score important segments in educational videos, leveraging audio, visual, and textual features to improve content summarization.

Contribution

It presents a new annotated dataset and a multimodal neural architecture for importance prediction in educational videos, addressing a key challenge in automated video summarization.

Findings

Multimodal features improve importance prediction accuracy.

Visual and temporal information significantly impact results.

The proposed model outperforms unimodal approaches.

Abstract

Videos are a commonly-used type of content in learning during Web search. Many e-learning platforms provide quality content, but sometimes educational videos are long and cover many topics. Humans are good in extracting important sections from videos, but it remains a significant challenge for computers. In this paper, we address the problem of assigning importance scores to video segments, that is how much information they contain with respect to the overall topic of an educational video. We present an annotation tool and a new dataset of annotated educational videos collected from popular online learning platforms. Moreover, we propose a multimodal neural architecture that utilizes state-of-the-art audio, visual and textual features. Our experiments investigate the impact of visual and temporal information, as well as the combination of multimodal features on importance prediction.