Controllable joint noise reduction and hearing loss compensation using a differentiable auditory model

TL;DR

This paper presents a multi-task learning approach using a differentiable auditory model to jointly perform noise reduction and hearing loss compensation, allowing flexible balancing between the two during inference.

Contribution

It introduces a novel multi-task framework that simultaneously optimizes noise reduction and hearing loss compensation with a differentiable auditory model, enabling adjustable task trade-offs.

Findings

Achieves comparable performance to single-task systems on objective metrics.

Allows dynamic adjustment of noise reduction and hearing loss compensation balance during inference.

Demonstrates the effectiveness of multi-task learning with differentiable auditory models.

Abstract

Deep learning-based hearing loss compensation (HLC) seeks to enhance speech intelligibility and quality for hearing impaired listeners using neural networks. One major challenge of HLC is the lack of a ground-truth target. Recent works have used neural networks to emulate non-differentiable auditory peripheral models in closed-loop frameworks, but this approach lacks flexibility. Alternatively, differentiable auditory models allow direct optimization, yet previous studies focused on individual listener profiles, or joint noise reduction (NR) and HLC without balancing each task. This work formulates NR and HLC as a multi-task learning problem, training a system to simultaneously predict denoised and compensated signals from noisy speech and audiograms using a differentiable auditory model. Results show the system achieves similar objective metric performance to systems trained for each task separately, while being able to adjust the balance between NR and HLC during inference.