Acoustic-to-Articulatory Inversion of Clean Speech Using an MRI-Trained Model

TL;DR

This paper explores using clean speech recordings instead of MRI data for articulatory inversion, demonstrating comparable accuracy and enabling practical applications without MRI noise interference.

Contribution

It introduces a method to perform articulatory inversion using clean speech data trained on MRI-based models, broadening practical usability.

Findings

Clean speech achieves RMSE of 1.56 mm in inversion accuracy.

Models trained on MRI data generalize well to clean speech.

Using clean speech simplifies data acquisition for articulatory inversion.

Abstract

Articulatory acoustic inversion reconstructs vocal tract shapes from speech. Real-time magnetic resonance imaging (rt-MRI) allows simultaneous acquisition of both the acoustic speech signal and articulatory information. Besides the complexity of rt-MRI acquisition, the recorded audio is heavily corrupted by scanner noise and requires denoising to be usable. For practical use, it must be possible to invert speech recorded without MRI noise. In this study, we investigate the use of speech recorded in a clean acoustic environment as an alternative to denoised MRI speech. To this end we compare two signals from the same speaker with identical sentences which are aligned using phonetic segmentation. A model trained on denoised MRI speech is evaluated on both denoised MRI and clean speech. We also assess a model trained and tested only on clean speech. Results show that clean speech supports articulatory inversion effectively, achieving an RMSE of 1.56 mm, close to MRI-based performance.