AMU-EURANOVA at CASE 2021 Task 1: Assessing the stability of multilingual BERT

TL;DR

This paper details the fine-tuning of multilingual BERT for event extraction in news, addressing stability issues due to limited training data in a shared task setting.

Contribution

It introduces an analysis of instability problems in multilingual BERT when applied to low-resource event extraction tasks and proposes mitigation strategies.

Findings

Identified instability issues in multilingual BERT with small datasets

Demonstrated improved stability with specific fine-tuning techniques

Achieved competitive performance in the shared task

Abstract

This paper explains our participation in task 1 of the CASE 2021 shared task. This task is about multilingual event extraction from news. We focused on sub-task 4, event information extraction. This sub-task has a small training dataset and we fine-tuned a multilingual BERT to solve this sub-task. We studied the instability problem on the dataset and tried to mitigate it.