Statistical Blendshape Calculation and Analysis for Graphics Applications

TL;DR

This paper presents a real-time, low-resource blendshape prediction system for facial animation using standard webcams, achieving accuracy comparable to advanced AR systems and suitable for VR applications.

Contribution

The authors developed an accurate, low-power blendshape prediction method for facial animation that operates in real-time with minimal hardware requirements.

Findings

Achieved ARKit 6-level accuracy in blendshape prediction

Ensured real-time response with low computational resources

Provided stable, smooth facial animation in VR applications

Abstract

With the development of virtualization and AI, real-time facial avatar animation is widely used in entertainment, office, business and other fields. Against this background, blendshapes have become a common industry animation solution because of their relative simplicity and ease of interpretation. Aiming for real-time performance and low computing resource dependence, we independently developed an accurate blendshape prediction system for low-power VR applications using a standard webcam. First, blendshape feature vectors are extracted through affine transformation and segmentation. Through further transformation and regression analysis, we were able to identify models for most blendshapes with significant predictive power. Post-processing was used to further improve response stability, including smoothing filtering and nonlinear transformations to minimize error. Experiments showed the system achieved accuracy similar to ARKit 6. Our model has low sensor/hardware requirements and realtime response with a consistent, accurate and smooth visual experience.