Text normalization using memory augmented neural networks

TL;DR

This paper introduces a memory-augmented neural network for text normalization that reduces errors, requires less data and training time, and is language-agnostic, outperforming traditional LSTM-based models.

Contribution

The paper presents a novel memory-augmented neural architecture for text normalization that improves accuracy and efficiency over existing models, with added capabilities for data up-sampling and meta-learning.

Findings

Reduces normalization errors compared to LSTM models.

Requires less data, training time, and computational resources.

Data up-sampling improves performance in semiotic classes.

Abstract

We perform text normalization, i.e. the transformation of words from the written to the spoken form, using a memory augmented neural network. With the addition of dynamic memory access and storage mechanism, we present a neural architecture that will serve as a language-agnostic text normalization system while avoiding the kind of unacceptable errors made by the LSTM-based recurrent neural networks. By successfully reducing the frequency of such mistakes, we show that this novel architecture is indeed a better alternative. Our proposed system requires significantly lesser amounts of data, training time and compute resources. Additionally, we perform data up-sampling, circumventing the data sparsity problem in some semiotic classes, to show that sufficient examples in any particular class can improve the performance of our text normalization system. Although a few occurrences of these errors still remain in certain semiotic classes, we demonstrate that memory augmented networks with meta-learning capabilities can open many doors to a superior text normalization system.