OCHADAI at SemEval-2022 Task 2: Adversarial Training for Multilingual Idiomaticity Detection

TL;DR

This paper introduces a multilingual adversarial training approach using transformer models for idiomaticity detection, effectively improving generalization without extra resources, and achieving competitive results in SemEval-2022.

Contribution

The paper presents a novel adversarial training method with multilingual transformers for idiomaticity detection, avoiding reliance on handcrafted features or external datasets.

Findings

Achieved 6th place in zero-shot setting

Achieved 15th place in one-shot setting

Demonstrated effectiveness of adversarial training with multilingual models

Abstract

We propose a multilingual adversarial training model for determining whether a sentence contains an idiomatic expression. Given that a key challenge with this task is the limited size of annotated data, our model relies on pre-trained contextual representations from different multi-lingual state-of-the-art transformer-based language models (i.e., multilingual BERT and XLM-RoBERTa), and on adversarial training, a training method for further enhancing model generalization and robustness. Without relying on any human-crafted features, knowledge bases, or additional datasets other than the target datasets, our model achieved competitive results and ranked 6th place in SubTask A (zero-shot) setting and 15th place in SubTask A (one-shot) setting.