A Regression Model of Recurrent Deep Neural Networks for Noise Robust Estimation of the Fundamental Frequency Contour of Speech

TL;DR

This paper introduces a regression-based deep neural network approach for estimating the fundamental frequency contour of speech, achieving higher accuracy in noisy conditions compared to existing classification methods.

Contribution

It presents a novel regression model using RNNs for F0 estimation, improving noise robustness and frequency resolution over prior classification-based methods.

Findings

Improves gross pitch error (GPE) by over 25% in noisy environments.

Enhances fine pitch error (FPE) by approximately 20% over state-of-the-art approaches.

Demonstrates effectiveness on noisy speech using PTDB-TUG corpus with additive noise.

Abstract

The fundamental frequency (F0) contour of speech is a key aspect to represent speech prosody that finds use in speech and spoken language analysis such as voice conversion and speech synthesis as well as speaker and language identification. This work proposes new methods to estimate the F0 contour of speech using deep neural networks (DNNs) and recurrent neural networks (RNNs). They are trained using supervised learning with the ground truth of F0 contours. The latest prior research addresses this problem first as a frame-by-frame-classification problem followed by sequence tracking using deep neural network hidden Markov model (DNN-HMM) hybrid architecture. This study, however, tackles the problem as a regression problem instead, in order to obtain F0 contours with higher frequency resolution from clean and noisy speech. Experiments using PTDB-TUG corpus contaminated with additive noise (NOISEX-92) show the proposed method improves gross pitch error (GPE) by more than 25 % at signal-to-noise ratios (SNRs) between -10 dB and +10 dB as compared with one of the most noise-robust F0 trackers, PEFAC. Furthermore, the performance on fine pitch error (FPE) is improved by approximately 20 % against a state-of-the-art DNN-HMM-based approach.