Semantic-Guided Unsupervised Video Summarization

TL;DR

This paper introduces a semantic-guided unsupervised video summarization method that leverages semantic alignment and incremental training to improve keyframe selection and address GAN training instability.

Contribution

It proposes a novel semantic alignment attention mechanism and an incremental training strategy within an adversarial framework for better video summarization.

Findings

Achieves superior performance on benchmark datasets.

Effectively guides keyframe selection with semantic information.

Reduces training instability in GAN-based models.

Abstract

Video summarization is a crucial technique for social understanding, enabling efficient browsing of massive multimedia content and extraction of key information from social platforms. Most existing unsupervised summarization methods rely on Generative Adversarial Networks (GANs) to enhance keyframe selection and generate coherent, video summaries through adversarial training. However, such approaches primarily exploit unimodal features, overlooking the guiding role of semantic information in keyframe selection, and often suffer from unstable training. To address these limitations, we propose a novel Semantic-Guided Unsupervised Video Summarization method. Specifically, we design a novel frame-level semantic alignment attention mechanism and integrate it into a keyframe selector, which guides the Transformer-based generator within the adversarial framework to better reconstruct videos. In addition, we adopt an incremental training strategy to progressively update the model components, effectively mitigating the instability of GAN training. Experimental results demonstrate that our approach achieves superior performance on multiple benchmark datasets.