A dataset and model for auditory scene recognition for hearing devices: AHEAD-DS and OpenYAMNet

TL;DR

This paper introduces AHEAD-DS, a standardized dataset for auditory scene recognition tailored for hearing devices, and OpenYAMNet, a lightweight model capable of real-time scene recognition on edge devices like smartphones.

Contribution

It provides a publicly accessible dataset with relevant labels and a deployable sound recognition model optimized for resource-constrained hearing aid applications.

Findings

OpenYAMNet achieved 0.86 mean average precision and 0.93 accuracy.

The model demonstrated real-time scene recognition on a Google Pixel 3.

Latency was approximately 50ms for model loading and 30ms per second of audio.

Abstract

Scene recognition is important for hearing devices, however; this is challenging, in part because of the limitations of existing datasets. Datasets often lack public accessibility, completeness, or audiologically relevant labels, hindering systematic comparison of machine learning models. Deploying such models on resource-constrained edge devices presents another challenge.The proposed solution is two-fold, a repack and refinement of several open source datasets to create AHEAD-DS, a dataset designed for auditory scene recognition for hearing devices, and introduce OpenYAMNet, a sound recognition model. AHEAD-DS aims to provide a standardised, publicly available dataset with consistent labels relevant to hearing aids, facilitating model comparison. OpenYAMNet is designed for deployment on edge devices like smartphones connected to hearing devices, such as hearing aids and wireless earphones with hearing aid functionality, serving as a baseline model for sound-based scene recognition. OpenYAMNet achieved a mean average precision of 0.86 and accuracy of 0.93 on the testing set of AHEAD-DS across fourteen categories relevant to auditory scene recognition. Real-time sound-based scene recognition capabilities were demonstrated on edge devices by deploying OpenYAMNet to an Android smartphone. Even with a 2018 Google Pixel 3, a phone with modest specifications, the model processes audio with approximately 50ms of latency to load the model, and an approximate linear increase of 30ms per 1 second of audio. The project website with links to code, data, and models. https://github.com/Australian-Future-Hearing-Initiative