# Improving Unsupervised Subword Modeling via Disentangled Speech Representation Learning and Transformation

## TL;DR

This paper introduces a method to improve unsupervised subword modeling by disentangling linguistic content and speaker information using FHVAE, leading to more accurate frame labels and better speech representations in zero-resource scenarios.

## Contribution

The study proposes integrating FHVAE with DNN-BNF to enhance frame label quality and speaker invariance in unsupervised speech modeling, demonstrating significant error rate reductions.

## Key findings

- Achieved 2.4% and 0.6% ABX error rate reductions in zero-resource speech tasks.
- Outperformed vocal tract length normalization in improving subword modeling.
- Disentangling speaker and linguistic features improves unsupervised speech representations.

## Abstract

This study tackles unsupervised subword modeling in the zero-resource scenario, learning frame-level speech representation that is phonetically discriminative and speaker-invariant, using only untranscribed speech for target languages. Frame label acquisition is an essential step in solving this problem. High quality frame labels should be in good consistency with golden transcriptions and robust to speaker variation. We propose to improve frame label acquisition in our previously adopted deep neural network-bottleneck feature (DNN-BNF) architecture by applying the factorized hierarchical variational autoencoder (FHVAE). FHVAEs learn to disentangle linguistic content and speaker identity information encoded in speech. By discarding or unifying speaker information, speaker-invariant features are learned and fed as inputs to DPGMM frame clustering and DNN-BNF training. Experiments conducted on ZeroSpeech 2017 show that our proposed approaches achieve $2.4\%$ and $0.6\%$ absolute ABX error rate reductions in across- and within-speaker conditions, comparing to the baseline DNN-BNF system without applying FHVAEs. Our proposed approaches significantly outperform vocal tract length normalization in improving frame labeling and subword modeling.