CustomCrafter: Customized Video Generation with Preserving Motion and Concept Composition Abilities
Tao Wu, Yong Zhang, Xintao Wang, Xianpan Zhou, Guangcong Zheng,, Zhongang Qi, Ying Shan, Xi Li

TL;DR
CustomCrafter introduces a framework that maintains motion and concept combination abilities in video diffusion models without additional videos or fine-tuning, enabling high-quality, customized video generation guided by text and reference images.
Contribution
It proposes a plug-and-play module for concept preservation and a dynamic sampling strategy to balance motion and appearance fidelity without extra data or re-tuning.
Findings
Significant improvement over previous methods in customized video generation.
Effective preservation of motion and concept abilities without additional videos.
High-quality, subject-specific videos generated with preserved motion and appearance.
Abstract
Customized video generation aims to generate high-quality videos guided by text prompts and subject's reference images. However, since it is only trained on static images, the fine-tuning process of subject learning disrupts abilities of video diffusion models (VDMs) to combine concepts and generate motions. To restore these abilities, some methods use additional video similar to the prompt to fine-tune or guide the model. This requires frequent changes of guiding videos and even re-tuning of the model when generating different motions, which is very inconvenient for users. In this paper, we propose CustomCrafter, a novel framework that preserves the model's motion generation and conceptual combination abilities without additional video and fine-tuning to recovery. For preserving conceptual combination ability, we design a plug-and-play module to update few parameters in VDMs, enhancing…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsVideo Analysis and Summarization · Human Motion and Animation · Artificial Intelligence in Games
MethodsDiffusion
