Multimodal LLMs are not all you need for Pediatric Speech Language Pathology

TL;DR

This paper presents a hierarchical, data-augmented approach using Speech Representation Models that outperforms LLMs in pediatric speech disorder classification tasks, addressing staffing shortages in speech therapy.

Contribution

It introduces a cascading classification method with targeted data augmentation for speech models, improving clinical task performance over existing LLM-based methods.

Findings

SRM outperforms LLMs across all tasks in the benchmark.

Data augmentation mitigates biases and enhances model robustness.

Hierarchical classification improves specificity in SSD diagnosis.

Abstract

Speech Sound Disorders (SSD) affect roughly five percent of children, yet speech-language pathologists face severe staffing shortages and unmanageable caseloads. We test a hierarchical approach to SSD classification on the granular multi-task SLPHelmUltraSuitePlus benchmark. We propose a cascading approach from binary classification to type, and symptom classification. By fine-tuning Speech Representation Models (SRM), and using targeted data augmentation we mitigate biases found by previous works, and improve upon all clinical tasks in the benchmark. We also treat Automatic Speech Recognition (ASR) with our data augmentation approach. Our results demonstrate that SRM consistently outperform the LLM-based state-of-the-art across all evaluated tasks by a large margin. We publish our models and code to foster future research.