A Survey: Spatiotemporal Consistency in Video Generation

TL;DR

This survey reviews recent advancements in video generation focusing on maintaining spatiotemporal consistency, discussing models, techniques, benchmarks, and future challenges to improve the quality and coherence of generated videos.

Contribution

It provides a comprehensive systematic review of methods, frameworks, and evaluation metrics for spatiotemporal consistency in video generation, highlighting current progress and future directions.

Findings

Analysis of various generation models and their effectiveness

Comparison of feature representations and training strategies

Identification of key challenges and promising research directions

Abstract

Video generation aims to produce temporally coherent sequences of visual frames, representing a pivotal advancement in Artificial Intelligence Generated Content (AIGC). Compared to static image generation, video generation poses unique challenges: it demands not only high-quality individual frames but also strong temporal coherence to ensure consistency throughout the spatiotemporal sequence. Although research addressing spatiotemporal consistency in video generation has increased in recent years, systematic reviews focusing on this core issue remain relatively scarce. To fill this gap, this paper views the video generation task as a sequential sampling process from a high-dimensional spatiotemporal distribution, and further discusses spatiotemporal consistency. We provide a systematic review of the latest advancements in the field. The content spans multiple dimensions including generation models, feature representations, generation frameworks, post-processing techniques, training strategies, benchmarks and evaluation metrics, with a particular focus on the mechanisms and effectiveness of various methods in maintaining spatiotemporal consistency. Finally, this paper explores future research directions and potential challenges in this field, aiming to provide valuable insights for advancing video generation technology. The project link is https://github.com/Yin-Z-Y/A-Survey-Spatiotemporal-Consistency-in-Video-Generation.