Learning the Cue or Learning the Word? Analyzing Generalization in Metaphor Detection for Verbs

TL;DR

This study investigates whether metaphor detection models generalize through learned contextual cues or lexical memorization, finding that context alone enables robust generalization.

Contribution

The paper introduces a controlled lexical hold-out setup and demonstrates that models primarily generalize via contextual cues rather than verb-specific memorization.

Findings

Models perform well on held-out verbs using sentence context.

Sentence context alone suffices for generalization, static verb embeddings do not.

Generalization is mainly driven by learning transferable contextual cues.

Abstract

Metaphor detection models achieve strong benchmark performance, yet it remains unclear whether this reflects transferable generalization or lexical memorization. To address this, we analyze generalization in metaphor detection through RoBERTa, the shared backbone of many state-of-the-art systems, focusing on English verbs using the VU Amsterdam Metaphor Corpus. We introduce a controlled lexical hold-out setup where all instances of selected target lemmas are strictly excluded from fine-tuning, and compare predictions on these Held-out lemmas against Exposed lemmas (verbs seen during fine-tuning). While the model performs best on Exposed lemmas, it maintains robust performance on Held-out lemmas. Further analysis reveals that sentence context alone is sufficient to match full-model performance on Held-out lemmas, whereas static verb-level embeddings are not. Together, these results suggest that generalization is primarily driven by "learning the cue" (transferable contextual patterns), while "learning the word" (verb-specific memorization) provides an additive boost when lexical exposure is available.