Towards objective and interpretable speech disorder assessment: a comparative analysis of CNN and transformer-based models

TL;DR

This paper compares CNN and transformer-based models for objective, interpretable assessment of speech disorders in head and neck cancer patients, demonstrating the effectiveness of self-supervised Wav2Vec2 models in phonetic feature discrimination.

Contribution

It introduces a self-supervised Wav2Vec2-based model for speech disorder assessment, highlighting its superior performance over CNNs and exploring factors affecting model accuracy.

Findings

Wav2Vec2 outperforms CNN in phonetic classification.

Model correlates well with perceptual speech measures.

Pre-training dataset choice impacts model performance.

Abstract

Head and Neck Cancers (HNC) significantly impact patients' ability to speak, affecting their quality of life. Commonly used metrics for assessing pathological speech are subjective, prompting the need for automated and unbiased evaluation methods. This study proposes a self-supervised Wav2Vec2-based model for phone classification with HNC patients, to enhance accuracy and improve the discrimination of phonetic features for subsequent interpretability purpose. The impact of pre-training datasets, model size, and fine-tuning datasets and parameters are explored. Evaluation on diverse corpora reveals the effectiveness of the Wav2Vec2 architecture, outperforming a CNN-based approach, used in previous work. Correlation with perceptual measures also affirms the model relevance for impaired speech analysis. This work paves the way for better understanding of pathological speech with interpretable approaches for clinicians, by leveraging complex self-learnt speech representations.