BLCU-ICALL at SemEval-2022 Task 1: Cross-Attention Multitasking Framework for Definition Modeling

TL;DR

This paper presents a transformer-based multitasking framework with cross-attention for definition modeling, achieving top rankings in SemEval-2022 Task 1 across multiple languages and demonstrating the effectiveness of ensemble strategies.

Contribution

It introduces a novel cross-attention multitasking framework that integrates multiple embeddings and captures gloss structure, advancing definition modeling techniques.

Findings

Achieved 1st place in Italian, 2nd in Spanish and Russian, 3rd in English and French.

Demonstrated the effectiveness of cross-attention and multitasking in definition modeling.

Showed that model ensembling improves robustness and performance.

Abstract

This paper describes the BLCU-ICALL system used in the SemEval-2022 Task 1 Comparing Dictionaries and Word Embeddings, the Definition Modeling subtrack, achieving 1st on Italian, 2nd on Spanish and Russian, and 3rd on English and French. We propose a transformer-based multitasking framework to explore the task. The framework integrates multiple embedding architectures through the cross-attention mechanism, and captures the structure of glosses through a masking language model objective. Additionally, we also investigate a simple but effective model ensembling strategy to further improve the robustness. The evaluation results show the effectiveness of our solution. We release our code at: https://github.com/blcuicall/SemEval2022-Task1-DM.