BERT(s) to Detect Multiword Expressions

TL;DR

This paper evaluates transformer-based models for detecting multiword expressions in NLP, demonstrating their superiority over previous LSTM-based neural models on a standard dataset.

Contribution

It provides an empirical comparison of transformer models for MWE detection, showing their improved performance over prior neural approaches.

Findings

Transformer models outperform LSTM-based models in MWE detection.

Empirical evaluation on SemEval-2016 dataset confirms transformer effectiveness.

Code and models will be publicly available.

Abstract

Multiword expressions (MWEs) present groups of words in which the meaning of the whole is not derived from the meaning of its parts. The task of processing MWEs is crucial in many natural language processing (NLP) applications, including machine translation and terminology extraction. Therefore, detecting MWEs is a popular research theme. In this paper, we explore state-of-the-art neural transformers in the task of detecting MWEs.We empirically evaluate several transformer models in the dataset for SemEval-2016 Task 10: Detecting Minimal Semantic Units and their Meanings (DiMSUM). We show that transformer models outperform the previous neural models based on long short-term memory (LSTM). The code and pre-trained model will be made freely available to the community.