A Chat About Boring Problems: Studying GPT-based text normalization

TL;DR

This paper demonstrates that Large-Language Models like GPT-3.5 and GPT-4 can effectively perform text normalization in few-shot settings, achieving significantly lower error rates than traditional systems by using innovative prompting and error analysis.

Contribution

It introduces a novel approach combining self-consistency and linguistic-informed prompts for LLM-based text normalization, and develops a new error taxonomy to analyze model performance.

Findings

LLMs achieve around 40% lower error rates than traditional systems.

Self-consistency reasoning improves normalization accuracy.

A new taxonomy reveals strengths and weaknesses of GPT-based normalization.

Abstract

Text normalization - the conversion of text from written to spoken form - is traditionally assumed to be an ill-formed task for language models. In this work, we argue otherwise. We empirically show the capacity of Large-Language Models (LLM) for text normalization in few-shot scenarios. Combining self-consistency reasoning with linguistic-informed prompt engineering, we find LLM based text normalization to achieve error rates around 40\% lower than top normalization systems. Further, upon error analysis, we note key limitations in the conventional design of text normalization tasks. We create a new taxonomy of text normalization errors and apply it to results from GPT-3.5-Turbo and GPT-4.0. Through this new framework, we can identify strengths and weaknesses of GPT-based TN, opening opportunities for future work.