# Are we there yet? Encoder-decoder neural networks as cognitive models of English past tense inflection

## TL;DR

This paper critically evaluates encoder-decoder neural networks as models for English past tense inflection, finding they lack consistent human-like performance and are not yet reliable cognitive models.

## Contribution

It provides a detailed analysis showing that current encoder-decoder models do not reliably replicate human data in past tense inflection tasks.

## Key findings

- Model exhibits instability across simulations.
- Aggregated results fit human data poorly.
- Performance remains inferior to older rule-based models.

## Abstract

The cognitive mechanisms needed to account for the English past tense have long been a subject of debate in linguistics and cognitive science. Neural network models were proposed early on, but were shown to have clear flaws. Recently, however, Kirov and Cotterell (2018) showed that modern encoder-decoder (ED) models overcome many of these flaws. They also presented evidence that ED models demonstrate humanlike performance in a nonce-word task. Here, we look more closely at the behaviour of their model in this task. We find that (1) the model exhibits instability across multiple simulations in terms of its correlation with human data, and (2) even when results are aggregated across simulations (treating each simulation as an individual human participant), the fit to the human data is not strong---worse than an older rule-based model. These findings hold up through several alternative training regimes and evaluation measures. Although other neural architectures might do better, we conclude that there is still insufficient evidence to claim that neural nets are a good cognitive model for this task.