Evaluating the cognitive reality of Spanish irregular morphomic patterns: Humans vs. Transformers

TL;DR

This study compares transformer models to human behavior in recognizing Spanish irregular morphomic patterns, revealing models' higher accuracy but differing preferences and limited phonological generalization, highlighting differences in linguistic generalization.

Contribution

It provides a direct comparison between transformer models and humans on Spanish irregular morphomic patterns using the same analytical framework.

Findings

Transformers outperform humans in stem accuracy.

Humans prefer natural inflections, models prefer irregular forms.

Models trained on natural and low-frequency data show phonological sensitivity.

Abstract

Do transformer models generalize morphological patterns like humans do? We investigate this by directly comparing transformers to human behavioral data on Spanish irregular morphomic patterns from \citet{Nevins2015TheRA}. We adopt the same analytical framework as the original human study. Under controlled input conditions, we evaluate whether transformer models can replicate human-like sensitivity to the morphome, a complex linguistic phenomenon. Our experiments focus on three frequency conditions: natural, low-frequency, and high-frequency distributions of verbs exhibiting irregular morphomic patterns. Transformer models achieve higher stem-accuracy than human participants. However, response preferences diverge: humans consistently favor the "natural" inflection across all items, whereas models preferred the irregular forms, and their choices are modulated by the proportion of irregular verbs present during training. Moreover, models trained on the natural and low-frequency distributions, but not the high-frequency distribution, exhibit sensitivity to phonological similarity between test items and Spanish L-shaped verbs, mirroring a limited aspect of human phonological generalization.