High-Quality Real Time Facial Capture Based on Single Camera
Hongwei Xu, Leijia Dai, Jianxing Fu, Xiangyuan Wang and, Quanwei Wang

TL;DR
This paper introduces a real-time deep learning system for facial expression capture from a single camera, enabling high-quality animation suitable for games and films with minimal manual effort.
Contribution
It presents a novel automated facial capture framework that leverages deep learning and high-end capture data to produce realistic blendshape outputs in real time.
Findings
Achieves high-quality facial animation in real time
Effectively captures challenging areas like eyes and lips
Reduces manual labor in digital character creation
Abstract
We propose a real time deep learning framework for video-based facial expression capture. Our process uses a high-end facial capture pipeline based on FACEGOOD to capture facial expression. We train a convolutional neural network to produce high-quality continuous blendshape weight output from video training. Since this facial capture is fully automated, our system can drastically reduce the amount of labor involved in the development of modern narrative-driven video games or films involving realistic digital doubles of actors and potentially hours of animated dialogue per character. We demonstrate compelling animation inference in challenging areas such as eyes and lips.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsFace recognition and analysis · Speech and Audio Processing · Human Motion and Animation
