MobilePortrait: Real-Time One-Shot Neural Head Avatars on Mobile Devices
Jianwen Jiang, Gaojie Lin, Zhengkun Rong, Chao Liang, Yongming Zhu,, Jiaqi Yang, Tianyun Zhong

TL;DR
MobilePortrait is a lightweight neural head avatar method enabling real-time, high-quality portrait animation on mobile devices by integrating external knowledge and efficient modeling techniques.
Contribution
It introduces a novel lightweight approach combining explicit and implicit keypoints with precomputed features for real-time mobile head avatar synthesis.
Findings
Achieves over 100 FPS on mobile devices.
Uses less than one-tenth the computational demand of prior methods.
Supports video and audio-driven inputs.
Abstract
Existing neural head avatars methods have achieved significant progress in the image quality and motion range of portrait animation. However, these methods neglect the computational overhead, and to the best of our knowledge, none is designed to run on mobile devices. This paper presents MobilePortrait, a lightweight one-shot neural head avatars method that reduces learning complexity by integrating external knowledge into both the motion modeling and image synthesis, enabling real-time inference on mobile devices. Specifically, we introduce a mixed representation of explicit and implicit keypoints for precise motion modeling and precomputed visual features for enhanced foreground and background synthesis. With these two key designs and using simple U-Nets as backbones, our method achieves state-of-the-art performance with less than one-tenth the computational demand. It has been…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsAugmented Reality Applications · Robotics and Automated Systems
