A Comparison of Discrete Latent Variable Models for Speech Representation Learning

TL;DR

This paper compares two neural latent variable models, vq-vae and vq-wav2vec, for speech representation learning, finding that vq-wav2vec with future prediction performs better on phoneme recognition tasks.

Contribution

It provides a systematic comparison of two prominent models, highlighting the advantages of future prediction-based approaches over auto-encoding in speech tasks.

Findings

vq-wav2vec outperforms vq-vae in phoneme recognition

Best system achieves 13.22 error rate on ZeroSpeech 2019 ABX challenge

Future time-step prediction enhances speech representation quality

Abstract

Neural latent variable models enable the discovery of interesting structure in speech audio data. This paper presents a comparison of two different approaches which are broadly based on predicting future time-steps or auto-encoding the input signal. Our study compares the representations learned by vq-vae and vq-wav2vec in terms of sub-word unit discovery and phoneme recognition performance. Results show that future time-step prediction with vq-wav2vec achieves better performance. The best system achieves an error rate of 13.22 on the ZeroSpeech 2019 ABX phoneme discrimination challenge