A cost minimization approach to fix the vocabulary size in a tokenizer for an End-to-End ASR system

TL;DR

This paper proposes a cost-based method to optimize vocabulary size in tokenization for end-to-end ASR systems, improving performance by selecting an optimal number of tokens.

Contribution

It introduces a novel cost function to determine the ideal vocabulary size, addressing the lack of systematic selection in existing tokenization approaches.

Findings

Optimized vocabulary size enhances ASR accuracy

Cost function effectively guides token count selection

Experimental results on LibriSpeech show performance gains

Abstract

Unlike hybrid speech recognition systems where the use of tokens was restricted to phones, biphones or triphones the choice of tokens in the end-to-end ASR systems is derived from the text corpus of the training data. The use of tokenization algorithms like Byte Pair Encoding (BPE) and WordPiece is popular in identifying the tokens that are used in the overall training process of the speech recognition system. Popular toolkits, like ESPNet use a pre-defined vocabulary size (number of tokens) for these tokenization algorithms, but there is no discussion on how vocabulary size was derived. In this paper, we build a cost function, assuming the tokenization process to be a black-box to enable choosing the number of tokens which might most benefit building an end-to-end ASR. We show through experiments on LibriSpeech 100 hour set that the performance of an end-to-end ASR system improves when the number of tokens are chosen carefully.