Training Strategies for Own Voice Reconstruction in Hearing Protection Devices using an In-ear Microphone

TL;DR

This paper explores training strategies for reconstructing broadband own voice speech from in-ear microphone recordings in noisy environments, emphasizing the use of simulated data and fine-tuning to address limited real data.

Contribution

It introduces a deep learning bandwidth-extension system and demonstrates effective training strategies combining simulated and real data for own voice reconstruction.

Findings

Simulated training data improves reconstruction performance.

Fine-tuning with real data enhances results.

Training on small real datasets is less effective.

Abstract

In-ear microphones in hearing protection devices can be utilized to capture the own voice speech of the person wearing the devices in noisy environments. Since in-ear recordings of the own voice are typically band-limited, an own voice reconstruction system is required to recover clean broadband speech from the in-ear signals. However, the availability of speech data for this scenario is typically limited due to device-specific transfer characteristics and the need to collect data from in-situ measurements. In this paper, we apply a deep learning-based bandwidth-extension system to the own voice reconstruction task and investigate different training strategies in order to overcome the limited availability of training data. Experimental results indicate that the use of simulated training data based on recordings of several talkers in combination with a fine-tuning approach using real data is advantageous compared to directly training on a small real dataset.