Query-Conditioned Three-Player Adversarial Network for Video Summarization

TL;DR

This paper introduces a novel query-conditioned three-player adversarial network for video summarization, effectively incorporating user queries to generate more personalized and relevant video summaries.

Contribution

It proposes a three-player GAN framework that jointly learns user query and video content representations, improving the quality of query-specific video summaries.

Findings

Outperforms existing methods on benchmark datasets

Effectively incorporates user queries for personalized summaries

Reduces trivial or irrelevant summaries

Abstract

Video summarization plays an important role in video understanding by selecting key frames/shots. Traditionally, it aims to find the most representative and diverse contents in a video as short summaries. Recently, a more generalized task, query-conditioned video summarization, has been introduced, which takes user queries into consideration to learn more user-oriented summaries. In this paper, we propose a query-conditioned three-player generative adversarial network to tackle this challenge. The generator learns the joint representation of the user query and the video content, and the discriminator takes three pairs of query-conditioned summaries as the input to discriminate the real summary from a generated and a random one. A three-player loss is introduced for joint training of the generator and the discriminator, which forces the generator to learn better summary results, and avoids the generation of random trivial summaries. Experiments on a recently proposed query-conditioned video summarization benchmark dataset show the efficiency and efficacy of our proposed method.