Multi-Microphone Noise Data Augmentation for DNN-based Own Voice Reconstruction for Hearables in Noisy Environments

TL;DR

This paper explores noise data augmentation techniques using measured transfer functions to improve deep learning-based own voice reconstruction in hearables, demonstrating that individualized noise augmentation significantly enhances system performance.

Contribution

It introduces novel noise augmentation methods based on measured transfer functions and shows their effectiveness in training more robust own voice reconstruction systems.

Findings

Augmented noise improves reconstruction accuracy.

Individualized noise augmentation yields higher performance.

The approach enhances generalization to real noisy environments.

Abstract

Hearables with integrated microphones may offer communication benefits in noisy working environments, e.g. by transmitting the recorded own voice of the user. Systems aiming at reconstructing the clean and full-bandwidth own voice from noisy microphone recordings are often based on supervised learning. Recording a sufficient amount of noise required for training such a system is costly since noise transmission between outer and inner microphones varies individually. Previously proposed methods either do not consider noise, only consider noise at outer microphones or assume inner and outer microphone noise to be independent during training, and it is not yet clear whether individualized noise can benefit the training of and own voice reconstruction system. In this paper, we investigate several noise data augmentation techniques based on measured transfer functions to simulate multi-microphone noise. Using augmented noise, we train a multi-channel own voice reconstruction system. Experiments using real noise are carried out to investigate the generalization capability. Results show that incorporating augmented noise yields large benefits, in particular considering individualized noise augmentation leads to higher performance.