Reconstruction of the Vocal Tract from Speech via Phonetic Representations Using MRI Data

TL;DR

This study compares different phonetic segmentation methods for reconstructing vocal tract geometry from speech signals, demonstrating that manual correction improves model accuracy and approaches baseline performance.

Contribution

It introduces a comparative analysis of phonetic segmentation accuracy levels in articulatory reconstruction, highlighting the benefits of manual correction over automatic methods.

Findings

Manual correction yields the best reconstruction performance.

Phonetic information improves articulatory contour prediction.

Approaches with manual correction approach baseline accuracy.

Abstract

Articulatory acoustic inversion aims to reconstruct the complete geometry of the vocal tract from the speech signal. In this paper, we present a comparative study of several levels of phonetic segmentation accuracy, together with a comparison to the baseline introduced in our previous work, which is based on Mel-Frequency Cepstral Coefficients (MFCCs). All the approaches considered are based on a denoised speech signal and aim to investigate the impact of incorporating phonetic information through three successive levels: an uncorrected automatic transcription, a temporally aligned phonetic segmentation, and an expert manual correction following alignment. The models are trained to predict articulatory contours extracted from vocal tract MRI images using an automatic contour tracking method. The results show that, among the models relying on phonetic representations, manual correction after alignment yields the best performance, approaching that of the baseline.