DreaMoving: A Human Video Generation Framework based on Diffusion Models

TL;DR

DreaMoving is a diffusion-based framework that generates high-quality, customizable human videos by controlling identity and motion sequences, enabling versatile and stylized video creation.

Contribution

It introduces a novel Video ControlNet and Content Guider for controllable, identity-preserving human video synthesis using diffusion models.

Findings

High-quality human video generation with controllable identity and motion.

Flexible adaptation to various diffusion models for diverse results.

Effective preservation of identity during motion-driven video synthesis.

Abstract

In this paper, we present DreaMoving, a diffusion-based controllable video generation framework to produce high-quality customized human videos. Specifically, given target identity and posture sequences, DreaMoving can generate a video of the target identity moving or dancing anywhere driven by the posture sequences. To this end, we propose a Video ControlNet for motion-controlling and a Content Guider for identity preserving. The proposed model is easy to use and can be adapted to most stylized diffusion models to generate diverse results. The project page is available at https://dreamoving.github.io/dreamoving