Enhancing Idiomatic Representation in Multiple Languages via an Adaptive Contrastive Triplet Loss

TL;DR

This paper introduces an adaptive contrastive triplet loss method to improve idiomatic expression modeling in multiple languages, addressing data scarcity and enhancing downstream NLP tasks.

Contribution

It proposes a novel triplet loss with adaptive contrastive learning and resampling miners to better capture idiomatic meanings in language models.

Findings

Outperforms previous methods on SemEval challenge

Significant improvements in idiomatic expression modeling metrics

Effective handling of non-compositional language in NLP

Abstract

Accurately modeling idiomatic or non-compositional language has been a longstanding challenge in Natural Language Processing (NLP). This is partly because these expressions do not derive their meanings solely from their constituent words, but also due to the scarcity of relevant data resources, and their impact on the performance of downstream tasks such as machine translation and simplification. In this paper we propose an approach to model idiomaticity effectively using a triplet loss that incorporates the asymmetric contribution of components words to an idiomatic meaning for training language models by using adaptive contrastive learning and resampling miners to build an idiomatic-aware learning objective. Our proposed method is evaluated on a SemEval challenge and outperforms previous alternatives significantly in many metrics.