Identifying Primary Stress Across Related Languages and Dialects with Transformer-based Speech Encoder Models

TL;DR

This paper demonstrates that fine-tuning a transformer-based speech encoder significantly improves primary stress detection across related languages and dialects, outperforming traditional acoustic feature methods.

Contribution

It introduces a transformer-based approach for primary stress identification and provides a new Croatian dataset with multilingual test sets, showing superior performance over traditional methods.

Findings

Transformer model outperforms SVM with acoustic features.

Near-perfect results for Croatian and Serbian.

Few hundred training words suffice for strong performance.

Abstract

Automating primary stress identification has been an active research field due to the role of stress in encoding meaning and aiding speech comprehension. Previous studies relied mainly on traditional acoustic features and English datasets. In this paper, we investigate the approach of fine-tuning a pre-trained transformer model with an audio frame classification head. Our experiments use a new Croatian training dataset, with test sets in Croatian, Serbian, the Chakavian dialect, and Slovenian. By comparing an SVM classifier using traditional acoustic features with the fine-tuned speech transformer, we demonstrate the transformer's superiority across the board, achieving near-perfect results for Croatian and Serbian, with a 10-point performance drop for the more distant Chakavian and Slovenian. Finally, we show that only a few hundred multi-syllabic training words suffice for strong performance. We release our datasets and model under permissive licenses.