Deep Learning For Prominence Detection In Children's Read Speech

TL;DR

This paper introduces a CRNN-based system using raw speech waveforms and SincNet filters for prominence detection in children's speech, enhancing oral fluency assessment with deep learning.

Contribution

It presents a novel deep learning approach operating directly on speech waveforms, integrating linguistic prosodic features and exploring multi-task architectures for prominence detection.

Findings

CRNN with SincNet filters improves prominence detection accuracy.

Multi-task learning with phrase boundary and prominence enhances performance.

Combining handcrafted acoustic and lexical features offers complementary benefits.

Abstract

The detection of perceived prominence in speech has attracted approaches ranging from the design of linguistic knowledge-based acoustic features to the automatic feature learning from suprasegmental attributes such as pitch and intensity contours. We present here, in contrast, a system that operates directly on segmented speech waveforms to learn features relevant to prominent word detection for children's oral fluency assessment. The chosen CRNN (convolutional recurrent neural network) framework, incorporating both word-level features and sequence information, is found to benefit from the perceptually motivated SincNet filters as the first convolutional layer. We further explore the benefits of the linguistic association between the prosodic events of phrase boundary and prominence with different multi-task architectures. Matching the previously reported performance on the same dataset of a random forest ensemble predictor trained on carefully chosen hand-crafted acoustic features, we evaluate further the possibly complementary information from hand-crafted acoustic and pre-trained lexical features.