Artimate: an articulatory animation framework for audiovisual speech synthesis

TL;DR

Artimate is a modular, open-source framework that animates the vocal tract for audiovisual speech synthesis using electromagnetic articulography data, enabling realistic tongue and teeth animation in virtual characters.

Contribution

It introduces a portable, open-source framework that applies EMA speech motion data to 3D vocal tract models for realistic articulatory animation in AV speech synthesis.

Findings

Provides realistic tongue and teeth animation for virtual characters

Integrates with 3D game engines for audiovisual applications

Uses open standards for portability and accessibility

Abstract

We present a modular framework for articulatory animation synthesis using speech motion capture data obtained with electromagnetic articulography (EMA). Adapting a skeletal animation approach, the articulatory motion data is applied to a three-dimensional (3D) model of the vocal tract, creating a portable resource that can be integrated in an audiovisual (AV) speech synthesis platform to provide realistic animation of the tongue and teeth for a virtual character. The framework also provides an interface to articulatory animation synthesis, as well as an example application to illustrate its use with a 3D game engine. We rely on cross-platform, open-source software and open standards to provide a lightweight, accessible, and portable workflow.