A Good Image Generator Is What You Need for High-Resolution Video Synthesis
Yu Tian, Jian Ren, Menglei Chai, Kyle Olszewski, Xi Peng, Dimitris N., Metaxas, Sergey Tulyakov

TL;DR
This paper introduces a novel framework that uses pre-trained image generators to efficiently produce high-resolution videos by discovering trajectories in the latent space, enabling content and motion manipulation and cross-domain synthesis.
Contribution
It presents a new method for high-resolution video synthesis leveraging image generators, including a motion generator for disentangled content and motion, and introduces cross-domain video synthesis.
Findings
Outperforms existing video generation methods in quality and efficiency
Enables content and motion manipulation in generated videos
Supports cross-domain video synthesis with disjoint datasets
Abstract
Image and video synthesis are closely related areas aiming at generating content from noise. While rapid progress has been demonstrated in improving image-based models to handle large resolutions, high-quality renderings, and wide variations in image content, achieving comparable video generation results remains problematic. We present a framework that leverages contemporary image generators to render high-resolution videos. We frame the video synthesis problem as discovering a trajectory in the latent space of a pre-trained and fixed image generator. Not only does such a framework render high-resolution videos, but it also is an order of magnitude more computationally efficient. We introduce a motion generator that discovers the desired trajectory, in which content and motion are disentangled. With such a representation, our framework allows for a broad range of applications, including…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
Taxonomy
TopicsGenerative Adversarial Networks and Image Synthesis · Advanced Vision and Imaging · Advanced Image Processing Techniques
