A Stacking Ensemble Approach for Supervised Video Summarization

TL;DR

This paper introduces a stacking ensemble approach that combines frame-level and shot-level methods for supervised video summarization, leading to improved performance over individual methods and state-of-the-art benchmarks.

Contribution

The paper proposes a novel stacking ensemble model that predicts key frames and interest segments simultaneously, enhancing video summarization accuracy.

Findings

Outperforms individual shot-level and frame-level methods.

Achieves superior results on benchmark datasets.

Demonstrates the effectiveness of soft decision fusion.

Abstract

Video summarization methods are usually classified into shot-level or frame-level methods, which are individually used in a general way. This paper investigates the underlying complementarity between the frame-level and shot-level methods, and a stacking ensemble approach is proposed for supervised video summarization. Firstly, we build up a stacking model to predict both the key frame probabilities and the temporal interest segments simultaneously. The two components are then combined via soft decision fusion to obtain the final scores of each frame in the video. A joint loss function is proposed for the model training. The ablation experimental results show that the proposed method outperforms both the two corresponding individual method. Furthermore, extensive experimental results on two benchmark datasets shows its superior performance in comparison with the state-of-the-art methods.