FlexiFilm: Long Video Generation with Flexible Conditions

TL;DR

FlexiFilm is a novel diffusion-based model designed for generating long, coherent videos over 30 seconds by introducing a temporal conditioner and resampling strategy to improve consistency and reduce overexposure.

Contribution

The paper presents FlexiFilm, a diffusion model specifically tailored for long video generation, addressing limitations of existing models in temporal consistency and overexposure.

Findings

Generates videos over 30 seconds long with improved temporal consistency.

Outperforms existing methods in qualitative and quantitative evaluations.

Effectively reduces overexposure in long video synthesis.

Abstract

Generating long and consistent videos has emerged as a significant yet challenging problem. While most existing diffusion-based video generation models, derived from image generation models, demonstrate promising performance in generating short videos, their simple conditioning mechanism and sampling strategy-originally designed for image generation-cause severe performance degradation when adapted to long video generation. This results in prominent temporal inconsistency and overexposure. Thus, in this work, we introduce FlexiFilm, a new diffusion model tailored for long video generation. Our framework incorporates a temporal conditioner to establish a more consistent relationship between generation and multi-modal conditions, and a resampling strategy to tackle overexposure. Empirical results demonstrate FlexiFilm generates long and consistent videos, each over 30 seconds in length, outperforming competitors in qualitative and quantitative analyses. Project page: https://y-ichen.github.io/FlexiFilm-Page/