Towards Improving the Expressiveness of Singing Voice Synthesis with BERT Derived Semantic Information

TL;DR

This paper enhances singing voice synthesis by integrating BERT-derived semantic embeddings, an energy predictor, and a re-designed pitch predictor, resulting in more expressive and natural singing voices with higher quality.

Contribution

It introduces the use of BERT-based semantic features and new predictors to improve expressiveness and stability in singing voice synthesis, surpassing previous models like VISinger.

Findings

Higher-quality singing voice synthesis demonstrated through experiments.

Semantic information from BERT improves expressiveness.

Energy and pitch predictors stabilize and enhance voice naturalness.

Abstract

This paper presents an end-to-end high-quality singing voice synthesis (SVS) system that uses bidirectional encoder representation from Transformers (BERT) derived semantic embeddings to improve the expressiveness of the synthesized singing voice. Based on the main architecture of recently proposed VISinger, we put forward several specific designs for expressive singing voice synthesis. First, different from the previous SVS models, we use text representation of lyrics extracted from pre-trained BERT as additional input to the model. The representation contains information about semantics of the lyrics, which could help SVS system produce more expressive and natural voice. Second, we further introduce an energy predictor to stabilize the synthesized voice and model the wider range of energy variations that also contribute to the expressiveness of singing voice. Last but not the least, to attenuate the off-key issues, the pitch predictor is re-designed to predict the real to note pitch ratio. Both objective and subjective experimental results indicate that the proposed SVS system can produce singing voice with higher-quality outperforming VISinger.