Align and Copy: UZH at SIGMORPHON 2017 Shared Task for Morphological Reinflection

TL;DR

This paper introduces neural network architectures with hard monotonic attention for morphological reinflection, achieving top results in a shared task, especially in limited-resource settings.

Contribution

The paper proposes two novel neural architectures with hard monotonic attention for morphological reinflection, excelling in low-resource scenarios and winning the shared task.

Findings

Best system overall in shared task

Outperforms competitors with only 100 training samples

Effective copying mechanisms in neural models

Abstract

This paper presents the submissions by the University of Zurich to the SIGMORPHON 2017 shared task on morphological reinflection. The task is to predict the inflected form given a lemma and a set of morpho-syntactic features. We focus on neural network approaches that can tackle the task in a limited-resource setting. As the transduction of the lemma into the inflected form is dominated by copying over lemma characters, we propose two recurrent neural network architectures with hard monotonic attention that are strong at copying and, yet, substantially different in how they achieve this. The first approach is an encoder-decoder model with a copy mechanism. The second approach is a neural state-transition system over a set of explicit edit actions, including a designated COPY action. We experiment with character alignment and find that naive, greedy alignment consistently produces strong results for some languages. Our best system combination is the overall winner of the SIGMORPHON 2017 Shared Task 1 without external resources. At a setting with 100 training samples, both our approaches, as ensembles of models, outperform the next best competitor.