Query-Aware Sparse Coding for Multi-Video Summarization

TL;DR

This paper introduces a query-aware multi-video summarization method using sparse coding, incorporating web images to capture user intent and organizing keyframes by event, validated on a new dataset.

Contribution

It presents a novel query-aware sparse coding framework for multi-video summarization that integrates web images and an event-based keyframe presentation structure.

Findings

Outperforms recent approaches in objective metrics.

Produces more relevant and user-friendly summaries.

Validated on the new MVS1K dataset with 1,000 videos.

Abstract

Given the explosive growth of online videos, it is becoming increasingly important to relieve the tedious work of browsing and managing the video content of interest. Video summarization aims at providing such a technique by transforming one or multiple videos into a compact one. However, conventional multi-video summarization methods often fail to produce satisfying results as they ignore the user's search intent. To this end, this paper proposes a novel query-aware approach by formulating the multi-video summarization in a sparse coding framework, where the web images searched by the query are taken as the important preference information to reveal the query intent. To provide a user-friendly summarization, this paper also develops an event-keyframe presentation structure to present keyframes in groups of specific events related to the query by using an unsupervised multi-graph fusion method. We release a new public dataset named MVS1K, which contains about 1, 000 videos from 10 queries and their video tags, manual annotations, and associated web images. Extensive experiments on MVS1K dataset validate our approaches produce superior objective and subjective results against several recently proposed approaches.