FreeTalk: Emotional Topology-Free 3D Talking Heads
Federico Nocentini, Thomas Besnier, Claudio Ferrari, Stefano Berretti, Mohamed Daoudi

TL;DR
FreeTalk is a novel framework for emotion-conditioned 3D talking-head animation that generalizes to arbitrary mesh topologies and captures both articulatory and emotional facial motions without relying on template meshes.
Contribution
It introduces a two-stage process combining sparse landmark prediction from speech with topology-free mesh deformation, enabling robust animation on raw 3D scans with arbitrary topology.
Findings
Matches specialized baselines in-domain
Improves robustness to unseen identities
Operates without template fitting or correspondence supervision
Abstract
Speech-driven 3D facial animation has advanced rapidly, yet most approaches remain tied to registered template meshes, preventing effective deployment on raw 3D scans with arbitrary topology. At the same time, modeling controllable emotional dynamics beyond lip articulation remains challenging, and is often tied to template-based parameterizations. We address these challenges by proposing FreeTalk, a two-stage framework for emotion-conditioned 3D talking-head animation that generalizes to unregistered face meshes with arbitrary vertex count and connectivity. First, Audio-To-Sparse (ATS) predicts a temporally coherent sequence of 3D landmark displacements from speech audio, conditioned on an emotion category and intensity. This sparse representation captures both articulatory and affective motion while remaining independent of mesh topology. Second, Sparse-To-Mesh (STM) transfers the…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsFace recognition and analysis · Generative Adversarial Networks and Image Synthesis · Emotion and Mood Recognition
