Score-Based Training for Energy-Based TTS Models

TL;DR

This paper introduces a novel training criterion for energy-based models that enhances score learning for better compatibility with first-order inference methods, addressing limitations of existing approaches like NCE and SSM.

Contribution

The paper proposes a new score-based training criterion tailored for energy-based models, improving their compatibility with first-order inference schemes.

Findings

The new criterion outperforms NCE and SSM in training EBMs.

Experimental results show improved inference quality with the proposed method.

The approach offers a more effective way to learn scores suitable for first-order optimization.

Abstract

Noise contrastive estimation (NCE) is a popular method for training energy-based models (EBM) with intractable normalisation terms. The key idea of NCE is to learn by comparing unnormalised log-likelihoods of the reference and noisy samples, thus avoiding explicitly computing normalisation terms. However, NCE critically relies on the quality of noisy samples. Recently, sliced score matching (SSM) has been popularised by closely related diffusion models (DM). Unlike NCE, SSM learns a gradient of log-likelihood, or score, by learning distribution of its projections on randomly chosen directions. However, both NCE and SSM disregard the form of log-likelihood function, which is problematic given that EBMs and DMs make use of first-order optimisation during inference. This paper proposes a new criterion that learns scores more suitable for first-order schemes. Experiments contrasts these approaches for training EBMs.