FG-Portrait: 3D Flow Guided Editable Portrait Animation
Yating Xu, Yunqi Miao, Evangelos Ververas, Jiankang Deng, Jifei Song

TL;DR
FG-Portrait introduces a geometry-driven, 3D flow-based approach for portrait animation that improves motion transfer accuracy and allows user editing, outperforming existing diffusion-based methods.
Contribution
The paper proposes a novel 3D flow encoding method using parametric head models, integrating geometry priors into diffusion models for improved portrait animation.
Findings
Outperforms existing methods in motion transfer accuracy
Preserves source identity more faithfully
Enables user-controlled facial editing
Abstract
Motion transfer from the driving to the source portrait remains a key challenge in the portrait animation. Current diffusion-based approaches condition only on the driving motion, which fails to capture source-to-driving correspondences and consequently yields suboptimal motion transfer. Although flow estimation provides an alternative, predicting dense correspondences from 2D input is ill-posed and often yields inaccurate animation. We address this problem by introducing 3D flows, a learning-free and geometry-driven motion correspondence directly computed from parametric 3D head models. To integrate this 3D prior into diffusion model, we introduce 3D flow encoding to query potential 3D flows for each target pixel to indicate its displacement back to the source location. To obtain 3D flows aligned with 2D motion changes, we further propose depth-guided sampling to accurately locate the…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsFace recognition and analysis · Generative Adversarial Networks and Image Synthesis · Computer Graphics and Visualization Techniques
