Disentangled-Transformer: An Explainable End-to-End Automatic Speech Recognition Model with Speech Content-Context Separation

TL;DR

This paper introduces the Disentangled-Transformer, an explainable end-to-end speech recognition model that separates speech content from speaker traits, enhancing interpretability and performance in speaker diarization.

Contribution

The study presents a novel transformer-based model that explicitly disentangles speech content and speaker traits, improving interpretability and diarization accuracy.

Findings

Effective separation of speaker identity from speech content.

Improved ASR performance with disentangled representations.

Enhanced interpretability of internal model representations.

Abstract

End-to-end transformer-based automatic speech recognition (ASR) systems often capture multiple speech traits in their learned representations that are highly entangled, leading to a lack of interpretability. In this study, we propose the explainable Disentangled-Transformer, which disentangles the internal representations into sub-embeddings with explicit content and speaker traits based on varying temporal resolutions. Experimental results show that the proposed Disentangled-Transformer produces a clear speaker identity, separated from the speech content, for speaker diarization while improving ASR performance.