VideoGigaGAN: Towards Detail-rich Video Super-Resolution

TL;DR

VideoGigaGAN is a novel generative model that extends image super-resolution techniques to videos, achieving high-frequency detail enhancement while maintaining strong temporal consistency, surpassing previous methods.

Contribution

The paper introduces VideoGigaGAN, a new VSR model that adapts large-scale image upsamplers with techniques to improve temporal consistency and detail in super-resolved videos.

Findings

Produces videos with high-frequency details and temporal consistency

Outperforms state-of-the-art VSR models on public datasets

Successfully achieves 8x super-resolution in experiments

Abstract

Video super-resolution (VSR) approaches have shown impressive temporal consistency in upsampled videos. However, these approaches tend to generate blurrier results than their image counterparts as they are limited in their generative capability. This raises a fundamental question: can we extend the success of a generative image upsampler to the VSR task while preserving the temporal consistency? We introduce VideoGigaGAN, a new generative VSR model that can produce videos with high-frequency details and temporal consistency. VideoGigaGAN builds upon a large-scale image upsampler -- GigaGAN. Simply inflating GigaGAN to a video model by adding temporal modules produces severe temporal flickering. We identify several key issues and propose techniques that significantly improve the temporal consistency of upsampled videos. Our experiments show that, unlike previous VSR methods, VideoGigaGAN generates temporally consistent videos with more fine-grained appearance details. We validate the effectiveness of VideoGigaGAN by comparing it with state-of-the-art VSR models on public datasets and showcasing video results with $8\times$ super-resolution.