RigAnyFace: Scaling Neural Facial Mesh Auto-Rigging with Unlabeled Data

TL;DR

RigAnyFace is a neural auto-rigging framework that effectively deforms diverse facial meshes, including disconnected components, using unlabeled data and a novel supervision strategy, outperforming previous methods in accuracy and generalization.

Contribution

The paper introduces a scalable neural auto-rigging method capable of handling complex topologies and disconnected facial components with unlabeled data, enhancing generalization and detail in facial animation.

Findings

Outperforms previous methods in accuracy and generalization.

Supports multiple disconnected facial components like eyeballs.

Uses unlabeled data with a novel supervision strategy to scale training.

Abstract

In this paper, we present RigAnyFace (RAF), a scalable neural auto-rigging framework for facial meshes of diverse topologies, including those with multiple disconnected components. RAF deforms a static neutral facial mesh into industry-standard FACS poses to form an expressive blendshape rig. Deformations are predicted by a triangulation-agnostic surface learning network augmented with our tailored architecture design to condition on FACS parameters and efficiently process disconnected components. For training, we curated a dataset of facial meshes, with a subset meticulously rigged by professional artists to serve as accurate 3D ground truth for deformation supervision. Due to the high cost of manual rigging, this subset is limited in size, constraining the generalization ability of models trained exclusively on it. To address this, we design a 2D supervision strategy for unlabeled neutral meshes without rigs. This strategy increases data diversity and allows for scaled training, thereby enhancing the generalization ability of models trained on this augmented data. Extensive experiments demonstrate that RAF is able to rig meshes of diverse topologies on not only our artist-crafted assets but also in-the-wild samples, outperforming previous works in accuracy and generalizability. Moreover, our method advances beyond prior work by supporting multiple disconnected components, such as eyeballs, for more detailed expression animation. Project page: https://wenchao-m.github.io/RigAnyFace.github.io