\'UFAL CorPipe at CRAC 2022: Effectivity of Multilingual Models for Coreference Resolution

TL;DR

This paper presents a multilingual coreference resolution system that fine-tunes large pretrained Transformer models, achieving state-of-the-art results across diverse languages and demonstrating the effectiveness of large multilingual models.

Contribution

The paper introduces a joint fine-tuning approach for mention detection and coreference linking using large multilingual Transformer models, showing improved performance across multiple languages.

Findings

Large multilingual models outperform smaller models on coreference tasks.

Fine-tuning with shared Transformer weights benefits all languages.

Performance gains are consistent across typologically diverse languages.

Abstract

We describe the winning submission to the CRAC 2022 Shared Task on Multilingual Coreference Resolution. Our system first solves mention detection and then coreference linking on the retrieved spans with an antecedent-maximization approach, and both tasks are fine-tuned jointly with shared Transformer weights. We report results of fine-tuning a wide range of pretrained models. The center of this contribution are fine-tuned multilingual models. We found one large multilingual model with sufficiently large encoder to increase performance on all datasets across the board, with the benefit not limited only to the underrepresented languages or groups of typologically relative languages. The source code is available at https://github.com/ufal/crac2022-corpipe.