A Multilinear Tongue Model Derived from Speech Related MRI Data of the Human Vocal Tract

TL;DR

This paper introduces a multilinear statistical model of the human tongue derived from MRI data, capturing anatomical and pose variations separately, enabling realistic tongue animation and registration of new data.

Contribution

The paper presents a novel multilinear tongue model derived from MRI data that effectively separates anatomical and speech-related shape variations.

Findings

Model reliably registers unknown data without overfitting.

Limiting degrees of freedom improves model robustness.

Can generate plausible tongue animations from sparse motion data.

Abstract

We present a multilinear statistical model of the human tongue that captures anatomical and tongue pose related shape variations separately. The model is derived from 3D magnetic resonance imaging data of 11 speakers sustaining speech related vocal tract configurations. The extraction is performed by using a minimally supervised method that uses as basis an image segmentation approach and a template fitting technique. Furthermore, it uses image denoising to deal with possibly corrupt data, palate surface information reconstruction to handle palatal tongue contacts, and a bootstrap strategy to refine the obtained shapes. Our evaluation concludes that limiting the degrees of freedom for the anatomical and speech related variations to 5 and 4, respectively, produces a model that can reliably register unknown data while avoiding overfitting effects. Furthermore, we show that it can be used to generate a plausible tongue animation by tracking sparse motion capture data.