Image Animation with Perturbed Masks
Yoav Shalev, Lior Wolf

TL;DR
This paper introduces a novel image animation method that animates objects without pose models, using perturbed masks and a shared mask generator, trained solely on reconstructing frames from the same video.
Contribution
The method animates arbitrary objects without prior pose models and only during test-time, outperforming state-of-the-art techniques on multiple benchmarks.
Findings
Outperforms existing methods on multiple benchmarks
Does not require pose models or prior structural knowledge
Works with arbitrary objects during test-time
Abstract
We present a novel approach for image-animation of a source image by a driving video, both depicting the same type of object. We do not assume the existence of pose models and our method is able to animate arbitrary objects without the knowledge of the object's structure. Furthermore, both, the driving video and the source image are only seen during test-time. Our method is based on a shared mask generator, which separates the foreground object from its background, and captures the object's general pose and shape. To control the source of the identity of the output frame, we employ perturbations to interrupt the unwanted identity information on the driver's mask. A mask-refinement module then replaces the identity of the driver with the identity of the source. Conditioned on the source image, the transformed mask is then decoded by a multi-scale generator that renders a realistic image,…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsAdvanced Vision and Imaging · Generative Adversarial Networks and Image Synthesis · Advanced Image and Video Retrieval Techniques
