NLP Datasets for Idiom and Figurative Language Tasks

TL;DR

This paper introduces new datasets for idiom and figurative language detection, aiding the development of NLP models that better understand informal and figurative expressions in large corpora.

Contribution

It provides a diverse set of datasets, including a large-scale idiom list and annotated datasets, to improve idiom recognition in NLP models.

Findings

Datasets enable better evaluation of idiom detection models.

Pre-trained models show limited ability to recognize idiomatic expressions.

Datasets facilitate training for improved figurative language understanding.

Abstract

Idiomatic and figurative language form a large portion of colloquial speech and writing. With social media, this informal language has become more easily observable to people and trainers of large language models (LLMs) alike. While the advantage of large corpora seems like the solution to all machine learning and Natural Language Processing (NLP) problems, idioms and figurative language continue to elude LLMs. Finetuning approaches are proving to be optimal, but better and larger datasets can help narrow this gap even further. The datasets presented in this paper provide one answer, while offering a diverse set of categories on which to build new models and develop new approaches. A selection of recent idiom and figurative language datasets were used to acquire a combined idiom list, which was used to retrieve context sequences from a large corpus. One large-scale dataset of potential idiomatic and figurative language expressions and two additional human-annotated datasets of definite idiomatic and figurative language expressions were created to evaluate the baseline ability of pre-trained language models in handling figurative meaning through idiom recognition (detection) tasks. The resulting datasets were post-processed for model agnostic training compatibility, utilized in training, and evaluated on slot labeling and sequence tagging.