Diversity Promoting Online Sampling for Streaming Video Summarization

TL;DR

This paper introduces a memory-efficient online sampling algorithm for streaming video summarization that balances clustering accuracy with sample diversity, outperforming traditional batch methods in both quality and resource usage.

Contribution

The paper presents a novel online algorithm based on competitive learning that promotes diversity in video sampling, reducing memory and computational needs compared to existing methods.

Findings

Outperforms batch summarization in quality

Requires significantly less memory and computation

Achieves better diversity in sampled frames

Abstract

Many applications benefit from sampling algorithms where a small number of well chosen samples are used to generalize different properties of a large dataset. In this paper, we use diverse sampling for streaming video summarization. Several emerging applications support streaming video, but existing summarization algorithms need access to the entire video which requires a lot of memory and computational power. We propose a memory efficient and computationally fast, online algorithm that uses competitive learning for diverse sampling. Our algorithm is a generalization of online K-means such that the cost function reduces clustering error, while also ensuring a diverse set of samples. The diversity is measured as the volume of a convex hull around the samples. Finally, the performance of the proposed algorithm is measured against human users for 50 videos in the VSUMM dataset. The algorithm performs better than batch mode summarization, while requiring significantly lower memory and computational requirements.