Lower Dimensional Kernels for Video Discriminators

TL;DR

This paper introduces lower-dimensional video discriminators for GANs that reduce optimization difficulty caused by high curvature loss surfaces, leading to improved performance on complex video datasets.

Contribution

It proposes a novel family of efficient lower-dimensional discriminators that enhance GAN training stability and performance on diverse video datasets.

Findings

Double the performance of Temporal-GANs

Achieve state-of-the-art results on UCF-101

Discriminators exhibit less curvature, easing optimization

Abstract

This work presents an analysis of the discriminators used in Generative Adversarial Networks (GANs) for Video. We show that unconstrained video discriminator architectures induce a loss surface with high curvature which make optimisation difficult. We also show that this curvature becomes more extreme as the maximal kernel dimension of video discriminators increases. With these observations in hand, we propose a family of efficient Lower-Dimensional Video Discriminators for GANs (LDVD GANs). The proposed family of discriminators improve the performance of video GAN models they are applied to and demonstrate good performance on complex and diverse datasets such as UCF-101. In particular, we show that they can double the performance of Temporal-GANs and provide for state-of-the-art performance on a single GPU.