Do Acoustic Word Embeddings Capture Phonological Similarity? An Empirical Study

TL;DR

This study investigates whether acoustic word embeddings truly reflect phonological similarity, revealing only moderate correlation and challenging current evaluation methods for AWEs.

Contribution

It provides an empirical analysis of neural AWE models across languages, showing their limitations in capturing phonological relationships.

Findings

Embedding distances only moderately correlate with phonological dissimilarity.

Improved word discrimination performance does not guarantee better phonological similarity modeling.

Current intrinsic evaluation methods for AWEs may need reconsideration.

Abstract

Several variants of deep neural networks have been successfully employed for building parametric models that project variable-duration spoken word segments onto fixed-size vector representations, or acoustic word embeddings (AWEs). However, it remains unclear to what degree we can rely on the distance in the emerging AWE space as an estimate of word-form similarity. In this paper, we ask: does the distance in the acoustic embedding space correlate with phonological dissimilarity? To answer this question, we empirically investigate the performance of supervised approaches for AWEs with different neural architectures and learning objectives. We train AWE models in controlled settings for two languages (German and Czech) and evaluate the embeddings on two tasks: word discrimination and phonological similarity. Our experiments show that (1) the distance in the embedding space in the best cases only moderately correlates with phonological distance, and (2) improving the performance on the word discrimination task does not necessarily yield models that better reflect word phonological similarity. Our findings highlight the necessity to rethink the current intrinsic evaluations for AWEs.