Learning to detect dysarthria from raw speech

TL;DR

This paper introduces a neural network that learns feature extraction, normalization, and compression directly from raw speech to improve dysarthria detection accuracy, surpassing traditional fixed features and prior learned features.

Contribution

It presents the first approach to jointly learn feature extraction, normalization, and compression directly from raw audio for speech classification tasks.

Findings

10% absolute accuracy improvement over fixed mel-filterbank features

Outperforms OpenSmile features when jointly learned from raw speech

Effective joint learning of multiple preprocessing steps from raw audio

Abstract

Speech classifiers of paralinguistic traits traditionally learn from diverse hand-crafted low-level features, by selecting the relevant information for the task at hand. We explore an alternative to this selection, by learning jointly the classifier, and the feature extraction. Recent work on speech recognition has shown improved performance over speech features by learning from the waveform. We extend this approach to paralinguistic classification and propose a neural network that can learn a filterbank, a normalization factor and a compression power from the raw speech, jointly with the rest of the architecture. We apply this model to dysarthria detection from sentence-level audio recordings. Starting from a strong attention-based baseline on which mel-filterbanks outperform standard low-level descriptors, we show that learning the filters or the normalization and compression improves over fixed features by 10% absolute accuracy. We also observe a gain over OpenSmile features by learning jointly the feature extraction, the normalization, and the compression factor with the architecture. This constitutes a first attempt at learning jointly all these operations from raw audio for a speech classification task.