Unifying Amplitude and Phase Analysis: A Compositional Data Approach to Functional Multivariate Mixed-Effects Modeling of Mandarin Chinese

TL;DR

This paper introduces a novel joint modeling approach combining functional data analysis, compositional data analysis, and linear mixed effects models to analyze pitch variations in Mandarin Chinese, accounting for amplitude and phase changes.

Contribution

It presents a unified model for amplitude, phase, and duration in speech analysis, integrating multiple statistical techniques for the first time in this context.

Findings

Phonetic information is jointly carried by amplitude and phase variations.

The model reveals dependence of pitch features on linguistic and non-linguistic covariates.

Application to Mandarin Chinese speech data demonstrates the model's effectiveness.

Abstract

Mandarin Chinese is characterized by being a tonal language; the pitch (or $F_0$) of its utterances carries considerable linguistic information. However, speech samples from different individuals are subject to changes in amplitude and phase which must be accounted for in any analysis which attempts to provide a linguistically meaningful description of the language. A joint model for amplitude, phase and duration is presented which combines elements from Functional Data Analysis, Compositional Data Analysis and Linear Mixed Effects Models. By decomposing functions via a functional principal component analysis, and connecting registration functions to compositional data analysis, a joint multivariate mixed effect model can be formulated which gives insights into the relationship between the different modes of variation as well as their dependence on linguistic and non-linguistic covariates. The model is applied to the COSPRO-1 data set, a comprehensive database of spoken Taiwanese Mandarin, containing approximately 50 thousand phonetically diverse sample $F_0$ contours (syllables), and reveals that phonetic information is jointly carried by both amplitude and phase variation.