Summary Transfer: Exemplar-based Subset Selection for Video Summarization

TL;DR

This paper introduces a novel exemplar-based subset selection method for video summarization that transfers summary structures from annotated videos to new videos, leveraging semantic information and extending to subshot-based summarization for efficiency.

Contribution

It presents a new transfer-based approach for video summarization that utilizes human-created summaries and semantic information, extending to subshot-level summarization for better efficiency.

Findings

Outperforms existing methods on multiple benchmarks.

Effective transfer of summary structures from annotated to new videos.

Extension to subshot-based summarization improves efficiency and flexibility.

Abstract

Video summarization has unprecedented importance to help us digest, browse, and search today's ever-growing video collections. We propose a novel subset selection technique that leverages supervision in the form of human-created summaries to perform automatic keyframe-based video summarization. The main idea is to nonparametrically transfer summary structures from annotated videos to unseen test videos. We show how to extend our method to exploit semantic side information about the video's category/genre to guide the transfer process by those training videos semantically consistent with the test input. We also show how to generalize our method to subshot-based summarization, which not only reduces computational costs but also provides more flexible ways of defining visual similarity across subshots spanning several frames. We conduct extensive evaluation on several benchmarks and demonstrate promising results, outperforming existing methods in several settings.