HRVGAN: High Resolution Video Generation using Spatio-Temporal GAN

TL;DR

This paper introduces HRVGAN, a novel high-resolution video generation framework that combines Wasserstein GANs with class conditioning to produce realistic, temporally coherent videos with improved quality and diversity.

Contribution

We propose a new deep generative network architecture for high-resolution video synthesis that integrates WGANs and cGANs, enhancing stability, semantic consistency, and visual quality.

Findings

Outperforms previous methods in quality and diversity metrics

Achieves high temporal coherence and spatial detail in generated videos

Validated on diverse datasets with superior quantitative results

Abstract

High-resolution video generation has emerged as a crucial task in computer vision, with wide-ranging applications in entertainment, simulation, and data augmentation. However, generating temporally coherent and visually realistic videos remains a significant challenge due to the high dimensionality and complex dynamics of video data. In this paper, we propose a novel deep generative network architecture designed specifically for high-resolution video synthesis. Our approach integrates key concepts from Wasserstein Generative Adversarial Networks (WGANs), enforcing a k-Lipschitz continuity constraint on the discriminator to stabilize training and enhance convergence. We further leverage Conditional GAN (cGAN) techniques by incorporating class labels during both training and inference, enabling class-specific video generation with improved semantic consistency. We provide a detailed layer-wise description of the Generator and Discriminator networks, highlighting architectural design choices promoting temporal coherence and spatial detail. The overall combined architecture, training algorithm, and optimization strategy are thoroughly presented. Our training objective combines a pixel-wise mean squared error loss with an adversarial loss to balance frame-level accuracy and video realism. We validate our approach on benchmark datasets including UCF101, Golf, and Aeroplane, encompassing diverse motion patterns and scene contexts. Quantitative evaluations using Inception Score (IS) and Fr\'echet Inception Distance (FID) demonstrate that our model significantly outperforms previous state-of-the-art unsupervised video generation methods in terms of both quality and diversity.