Unsupervised video summarization framework using keyframe extraction and video skimming

TL;DR

This paper presents an unsupervised framework for video summarization that combines traditional and deep learning-based feature extraction methods with clustering to identify keyframes, improving summaries especially for dynamic videos.

Contribution

It introduces a novel unsupervised approach integrating deep learning features and clustering for effective video summarization, reducing reliance on labeled data.

Findings

Deep learning features outperform traditional features in dynamic videos.

Clustering methods effectively identify keyframes for summarization.

Unsupervised approach reduces need for extensive labeled datasets.

Abstract

Video is one of the robust sources of information and the consumption of online and offline videos has reached an unprecedented level in the last few years. A fundamental challenge of extracting information from videos is a viewer has to go through the complete video to understand the context, as opposed to an image where the viewer can extract information from a single frame. Apart from context understanding, it almost impossible to create a universal summarized video for everyone, as everyone has their own bias of keyframe, e.g; In a soccer game, a coach person might consider those frames which consist of information on player placement, techniques, etc; however, a person with less knowledge about a soccer game, will focus more on frames which consist of goals and score-board. Therefore, if we were to tackle problem video summarization through a supervised learning path, it will require extensive personalized labeling of data. In this paper, we attempt to solve video summarization through unsupervised learning by employing traditional vision-based algorithmic methodologies for accurate feature extraction from video frames. We have also proposed a deep learning-based feature extraction followed by multiple clustering methods to find an effective way of summarizing a video by interesting key-frame extraction. We have compared the performance of these approaches on the SumMe dataset and showcased that using deep learning-based feature extraction has been proven to perform better in case of dynamic viewpoint videos.