Generative 3D Animation Pipelines: Automating Facial Retargeting Workflows

TL;DR

This paper explores integrating deepfake technology into 3D animation workflows to automate facial retargeting, reducing manual effort and improving efficiency, while addressing ethical concerns and design considerations.

Contribution

It introduces a novel approach to embedding deepfake algorithms into 3D animation pipelines for facial retargeting, with a prototype and expert feedback.

Findings

Positive expert feedback on the tool

Deepfake integration reduces manual retargeting effort

Design practices to prevent misuse

Abstract

Design tools in the 3D industry, while powerful, are still tedious and labor-intensive when it comes to bringing a creative idea for a visual effect to life. In this position paper, we discussed how an infamous generative synthetic media, deepfakes, could be of use and embedded into common sophisticated 3D workflows to reduce user workloads in areas such as 3D model editing, material design, and character animation. As a case discussion, we also prototyped a tool to address the retargeting problem in character animation. Although deepfakes themselves have received a negative public image, the results of our interviews with field experts are unexpectedly positive in regard to our tool that utilizes deepfake algorithms. Lastly, we also discussed our experience and observed design practices to put deepfakes to good use, including how we could avoid potential misuses directly by design, how this design changes user interactions, and subsequent open issues.