Text-Guided Synthesis of Eulerian Cinemagraphs
Aniruddha Mahapatra, Aliaksandr Siarohin, Hsin-Ying Lee, Sergey, Tulyakov, Jun-Yan Zhu

TL;DR
This paper presents Text2Cinemagraph, an automated method that generates artistic cinemagraphs from text prompts by synthesizing image pairs and transferring realistic motion to artistic images, improving consistency and artistic control.
Contribution
It introduces a novel approach of creating image twins from text prompts, enabling effective motion transfer for artistic and natural scenes in cinemagraph synthesis.
Findings
Outperforms existing methods in natural and artistic cinemagraphs
Achieves higher consistency and realism validated by metrics and user studies
Enables animation of paintings and control of motion directions
Abstract
We introduce Text2Cinemagraph, a fully automated method for creating cinemagraphs from text descriptions - an especially challenging task when prompts feature imaginary elements and artistic styles, given the complexity of interpreting the semantics and motions of these images. We focus on cinemagraphs of fluid elements, such as flowing rivers, and drifting clouds, which exhibit continuous motion and repetitive textures. Existing single-image animation methods fall short on artistic inputs, and recent text-based video methods frequently introduce temporal inconsistencies, struggling to keep certain regions static. To address these challenges, we propose an idea of synthesizing image twins from a single text prompt - a pair of an artistic image and its pixel-aligned corresponding natural-looking twin. While the artistic image depicts the style and appearance detailed in our text prompt,…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsVideo Analysis and Summarization · Human Motion and Animation · Generative Adversarial Networks and Image Synthesis
MethodsFocus
