Text-to-speech for the hearing impaired

TL;DR

This paper introduces an advanced TTS system that enhances speech loudness for the hearing impaired, improving intelligibility and quality through a novel loudness restoration algorithm and transfer learning techniques.

Contribution

It presents a new loudness restoration algorithm integrated into a TTS system using Tacotron2 and WaveGlow, enabling personalized amplification with high speech quality.

Findings

High speech quality comparable to original speech

Significantly improved speech intelligibility in noise

Effective transfer learning for quick individual adaptation

Abstract

Text-to-speech (TTS) systems offer the opportunity to compensate for a hearing loss at the source rather than correcting for it at the receiving end. This removes limitations such as time constraints for algorithms that amplify a sound in a hearing aid and can lead to higher speech quality. We propose an algorithm that restores loudness to normal perception at a high resolution in time, frequency and level, and embed it in a TTS system that uses Tacotron2 and WaveGlow to produce individually amplified speech. Subjective evaluations of speech quality showed that the proposed algorithm led to high-quality audio with sound quality similar to original or linearly amplified speech but considerably higher speech intelligibility in noise. Transfer learning led to a quick adaptation of the produced spectra from original speech to individually amplified speech, resulted in high speech quality and intelligibility, and thus gives us a way to train an individual TTS system efficiently.