DAT-CFTNet: Speech Enhancement for Cochlear Implant Recipients using Attention-based Dual-Path Recurrent Neural Network

TL;DR

This paper introduces DAT-CFTNet, a novel attention-based dual-path RNN integrated with CFTNet, significantly improving speech enhancement for cochlear implant users by effectively suppressing noise and preserving speech quality.

Contribution

It proposes a new attention mechanism within a dual-path RNN framework combined with CFTNet, enhancing speech intelligibility and quality for cochlear implant recipients in noisy environments.

Findings

Outperforms existing models like CFTNet and DCCRN in speech intelligibility and quality.

Effectively suppresses non-stationary noise and avoids musical artifacts.

Shows superior performance in cochlear implant speech enhancement studies.

Abstract

The human auditory system has the ability to selectively focus on key speech elements in an audio stream while giving secondary attention to less relevant areas such as noise or distortion within the background, dynamically adjusting its attention over time. Inspired by the recent success of attention models, this study introduces a dual-path attention module in the bottleneck layer of a concurrent speech enhancement network. Our study proposes an attention-based dual-path RNN (DAT-RNN), which, when combined with the modified complex-valued frequency transformation network (CFTNet), forms the DAT-CFTNet. This attention mechanism allows for precise differentiation between speech and noise in time-frequency (T-F) regions of spectrograms, optimizing both local and global context information processing in the CFTNet. Our experiments suggest that the DAT-CFTNet leads to consistently improved performance over the existing models, including CFTNet and DCCRN, in terms of speech intelligibility and quality. Moreover, the proposed model exhibits superior performance in enhancing speech intelligibility for cochlear implant (CI) recipients, who are known to have severely limited T-F hearing restoration (e.g., >10%) in CI listener studies in noisy settings show the proposed solution is capable of suppressing non-stationary noise, avoiding the musical artifacts often seen in traditional speech enhancement methods. The implementation of the proposed model will be publicly available.