Neural Induction of Finite-State Transducers

TL;DR

This paper introduces a neural method for automatically inducing finite-state transducers from RNNs, achieving high accuracy and robustness in string-to-string tasks like morphological inflection and phoneme prediction, outperforming classical algorithms.

Contribution

The paper presents a novel neural approach to construct unweighted FSTs based on hidden state geometry learned by RNNs, improving over traditional transducer learning methods.

Findings

Achieves up to 87% accuracy on test datasets

Outperforms classical transducer learning algorithms

Demonstrates robustness across multiple real-world tasks

Abstract

Finite-State Transducers (FSTs) are effective models for string-to-string rewriting tasks, often providing the efficiency necessary for high-performance applications, but constructing transducers by hand is difficult. In this work, we propose a novel method for automatically constructing unweighted FSTs following the hidden state geometry learned by a recurrent neural network. We evaluate our methods on real-world datasets for morphological inflection, grapheme-to-phoneme prediction, and historical normalization, showing that the constructed FSTs are highly accurate and robust for many datasets, substantially outperforming classical transducer learning algorithms by up to 87% accuracy on held-out test sets.