Compressed Skinning for Facial Blendshapes

TL;DR

This paper introduces a novel facial skinning compression method that uses proximal algorithms to produce sparse, efficient linear blend skinning representations, significantly reducing memory and computation costs while maintaining visual quality.

Contribution

It proposes a flexible proximal algorithm-based approach for skinning decomposition that enforces sparsity, enabling faster and more memory-efficient facial animation representations.

Findings

Achieves similar accuracy and visual quality as state-of-the-art methods.

Enforces approximately 10% non-zero transformation coefficients for sparsity.

Provides a PyTorch implementation for easy experimentation.

Abstract

We present a new method to bake classical facial animation blendshapes into a fast linear blend skinning representation. Previous work explored skinning decomposition methods that approximate general animated meshes using a dense set of bone transformations; these optimizers typically alternate between optimizing for the bone transformations and the skinning weights.We depart from this alternating scheme and propose a new approach based on proximal algorithms, which effectively means adding a projection step to the popular Adam optimizer. This approach is very flexible and allows us to quickly experiment with various additional constraints and/or loss functions. Specifically, we depart from the classical skinning paradigms and restrict the transformation coefficients to contain only about 10% non-zeros, while achieving similar accuracy and visual quality as the state-of-the-art. The sparse storage enables our method to deliver significant savings in terms of both memory and run-time speed. We include a compact implementation of our new skinning decomposition method in PyTorch, which is easy to experiment with and modify to related problems.