Avatar Fingerprinting for Authorized Use of Synthetic Talking-Head Videos

TL;DR

This paper introduces a novel method for avatar fingerprinting to verify the authenticity of synthetic talking-head videos, using a large dataset and motion signatures that distinguish real from synthetic content with high accuracy.

Contribution

It presents the first avatar fingerprinting technique that generalizes to unseen generators, supported by a new large-scale dataset of real and synthetic videos.

Findings

Achieves an average AUC of 0.85 in identifying synthetic videos.

Generalizes well to new, unseen generators with an AUC of 0.83.

Provides a large dataset for future research in avatar verification.

Abstract

Modern avatar generators allow anyone to synthesize photorealistic real-time talking avatars, ushering in a new era of avatar-based human communication, such as with immersive AR/VR interactions or videoconferencing with limited bandwidths. Their safe adoption, however, requires a mechanism to verify if the rendered avatar is trustworthy: does it use the appearance of an individual without their consent? We term this task avatar fingerprinting. To tackle it, we first introduce a large-scale dataset of real and synthetic videos of people interacting on a video call, where the synthetic videos are generated using the facial appearance of one person and the expressions of another. We verify the identity driving the expressions in a synthetic video, by learning motion signatures that are independent of the facial appearance shown. Our solution, the first in this space, achieves an average AUC of 0.85. Critical to its practical use, it also generalizes to new generators never seen in training (average AUC of 0.83). The proposed dataset and other resources can be found at: https://research.nvidia.com/labs/nxp/avatar-fingerprinting/.