Enhancing Child Vocalization Classification with Phonetically-Tuned Embeddings for Assisting Autism Diagnosis

TL;DR

This paper introduces a phonetically-tuned embedding approach using Wav2Vec 2.0 to improve child vocalization classification, aiding autism diagnosis by enhancing speaker diarization and vocalization detection in child audio recordings.

Contribution

It develops a child-specific phoneme recognition system and integrates its embeddings into Wav2Vec 2.0, achieving improved vocalization classification and outperforming state-of-the-art methods.

Findings

Consistent performance improvements on Rapid-ABC and BabbleCor datasets.

Outperforms previous state-of-the-art on BabbleCor subset.

Effective use of phonetic embeddings enhances child vocalization detection.

Abstract

The assessment of children at risk of autism typically involves a clinician observing, taking notes, and rating children's behaviors. A machine learning model that can label adult and child audio may largely save labor in coding children's behaviors, helping clinicians capture critical events and better communicate with parents. In this study, we leverage Wav2Vec 2.0 (W2V2), pre-trained on 4300-hour of home audio of children under 5 years old, to build a unified system for tasks of clinician-child speaker diarization and vocalization classification (VC). To enhance children's VC, we build a W2V2 phoneme recognition system for children under 4 years old, and we incorporate its phonetically-tuned embeddings as auxiliary features or recognize pseudo phonetic transcripts as an auxiliary task. We test our method on two corpora (Rapid-ABC and BabbleCor) and obtain consistent improvements. Additionally, we outperform the state-of-the-art performance on the reproducible subset of BabbleCor. Code available at https://huggingface.co/lijialudew