Text-Animator: Controllable Visual Text Video Generation

TL;DR

Text-Animator is a novel method for generating videos with visually accurate and stable embedded text, addressing the challenge of visualizing text in T2V generation by controlling camera and text motion.

Contribution

It introduces a text embedding injection, camera control, and text refinement modules to enhance text visualization and stability in generated videos.

Findings

Outperforms state-of-the-art methods in visual text accuracy

Improves stability of visualized text through camera and motion control

Demonstrates superior qualitative and quantitative results

Abstract

Video generation is a challenging yet pivotal task in various industries, such as gaming, e-commerce, and advertising. One significant unresolved aspect within T2V is the effective visualization of text within generated videos. Despite the progress achieved in Text-to-Video~(T2V) generation, current methods still cannot effectively visualize texts in videos directly, as they mainly focus on summarizing semantic scene information, understanding, and depicting actions. While recent advances in image-level visual text generation show promise, transitioning these techniques into the video domain faces problems, notably in preserving textual fidelity and motion coherence. In this paper, we propose an innovative approach termed Text-Animator for visual text video generation. Text-Animator contains a text embedding injection module to precisely depict the structures of visual text in generated videos. Besides, we develop a camera control module and a text refinement module to improve the stability of generated visual text by controlling the camera movement as well as the motion of visualized text. Quantitative and qualitative experimental results demonstrate the superiority of our approach to the accuracy of generated visual text over state-of-the-art video generation methods. The project page can be found at https://laulampaul.github.io/text-animator.html.