Speaker-independent Speech Inversion for Estimation of Nasalance

TL;DR

This paper investigates the relationship between nasalance and VP movement, demonstrating significant correlation, and introduces a speaker-independent speech inversion system that estimates VP movement from acoustic signals, improving nasality prediction.

Contribution

It presents the first evaluation of nasalance's reflection of VP movement and develops a novel speaker-independent model for estimating VP motion from speech signals.

Findings

Nasalance significantly correlates with VP opening measured by HSN.

Incorporating source features improves nasality prediction.

The proposed model effectively estimates VP movement from speech.

Abstract

The velopharyngeal (VP) valve regulates the opening between the nasal and oral cavities. This valve opens and closes through a coordinated motion of the velum and pharyngeal walls. Nasalance is an objective measure derived from the oral and nasal acoustic signals that correlate with nasality. In this work, we evaluate the degree to which the nasalance measure reflects fine-grained patterns of VP movement by comparison with simultaneously collected direct measures of VP opening using high-speed nasopharyngoscopy (HSN). We show that nasalance is significantly correlated with the HSN signal, and that both match expected patterns of nasality. We then train a temporal convolution-based speech inversion system in a speaker-independent fashion to estimate VP movement for nasality, using nasalance as the ground truth. In further experiments, we also show the importance of incorporating source features (from glottal activity) to improve nasality prediction.