Temporal Generative Adversarial Nets with Singular Value Clipping

TL;DR

This paper introduces Temporal Generative Adversarial Nets (TGAN), a novel video generation model that uses separate temporal and image generators, trained with Wasserstein GAN for stability, capable of learning semantic video representations.

Contribution

The paper presents a new TGAN architecture with separate generators for temporal and image features, and a stable training method using Wasserstein GAN, advancing video generation capabilities.

Findings

Effective video generation demonstrated in experiments

Stable end-to-end training achieved with Wasserstein GAN

Model captures semantic representations of unlabeled videos

Abstract

In this paper, we propose a generative model, Temporal Generative Adversarial Nets (TGAN), which can learn a semantic representation of unlabeled videos, and is capable of generating videos. Unlike existing Generative Adversarial Nets (GAN)-based methods that generate videos with a single generator consisting of 3D deconvolutional layers, our model exploits two different types of generators: a temporal generator and an image generator. The temporal generator takes a single latent variable as input and outputs a set of latent variables, each of which corresponds to an image frame in a video. The image generator transforms a set of such latent variables into a video. To deal with instability in training of GAN with such advanced networks, we adopt a recently proposed model, Wasserstein GAN, and propose a novel method to train it stably in an end-to-end manner. The experimental results demonstrate the effectiveness of our methods.