Binary Token-Level Classification with DeBERTa for All-Type MWE Identification: A Lightweight Approach with Linguistic Enhancement

TL;DR

This paper introduces a lightweight DeBERTa-based method for multiword expression identification that outperforms larger models by reformulating detection as token-level classification, integrating linguistic features, and using data augmentation.

Contribution

The approach reformulates MWE detection as token-level classification, incorporates linguistic features, and demonstrates high performance with a smaller model, outperforming larger language models.

Findings

Achieved 69.8% F1 on CoAM dataset, surpassing larger models.

Outperformed previous best by 12 points with 165x fewer parameters.

Confirmed generalization on STREUSLE dataset with 78.9% F1.

Abstract

We present a comprehensive approach for multiword expression (MWE) identification that combines binary token-level classification, linguistic feature integration, and data augmentation. Our DeBERTa-v3-large model achieves 69.8% F1 on the CoAM dataset, surpassing the best results (Qwen-72B, 57.8% F1) on this dataset by 12 points while using 165x fewer parameters. We achieve this performance by (1) reformulating detection as binary token-level START/END/INSIDE classification rather than span-based prediction, (2) incorporating NP chunking and dependency features that help discontinuous and NOUN-type MWEs identification, and (3) applying oversampling that addresses severe class imbalance in the training data. We confirm the generalization of our method on the STREUSLE dataset, achieving 78.9% F1. These results demonstrate that carefully designed smaller models can substantially outperform LLMs on structured NLP tasks, with important implications for resource-constrained deployments.