DTSGAN: Learning Dynamic Textures via Spatiotemporal Generative Adversarial Network

TL;DR

DTSGAN is a novel spatiotemporal GAN that learns from a single dynamic texture to generate diverse, high-quality video sequences with natural motion, advancing dynamic texture synthesis.

Contribution

Introduces DTSGAN, a new method for learning dynamic textures from a single example using a multi-scale pipeline and a data update strategy to enhance diversity.

Findings

Generates high-quality dynamic textures with natural motion

Outperforms existing methods in diversity and realism

Effective in learning from a single sample

Abstract

Dynamic texture synthesis aims to generate sequences that are visually similar to a reference video texture and exhibit specific stationary properties in time. In this paper, we introduce a spatiotemporal generative adversarial network (DTSGAN) that can learn from a single dynamic texture by capturing its motion and content distribution. With the pipeline of DTSGAN, a new video sequence is generated from the coarsest scale to the finest one. To avoid mode collapse, we propose a novel strategy for data updates that helps improve the diversity of generated results. Qualitative and quantitative experiments show that our model is able to generate high quality dynamic textures and natural motion.