Searching for Search Errors in Neural Morphological Inflection

TL;DR

This paper investigates why neural models for morphological inflection often produce empty strings during inference, revealing that the issue is task-specific rather than a general flaw of neural language models.

Contribution

The study demonstrates that the empty string problem is not due to neural model inadequacy but relates to specific characteristics of morphological inflection tasks.

Findings

Empty string is rarely the most probable output in morphological inflection.

Greedy search often finds the global optimum in these models.

Poor calibration of neural models may be task-dependent.

Abstract

Neural sequence-to-sequence models are currently the predominant choice for language generation tasks. Yet, on word-level tasks, exact inference of these models reveals the empty string is often the global optimum. Prior works have speculated this phenomenon is a result of the inadequacy of neural models for language generation. However, in the case of morphological inflection, we find that the empty string is almost never the most probable solution under the model. Further, greedy search often finds the global optimum. These observations suggest that the poor calibration of many neural models may stem from characteristics of a specific subset of tasks rather than general ill-suitedness of such models for language generation.