BUT-FIT at SemEval-2020 Task 5: Automatic detection of counterfactual statements with deep pre-trained language representation models

TL;DR

This paper presents a system using deep pre-trained language models, particularly RoBERTa, to detect counterfactual statements and extract their components, achieving top results in SemEval-2020 Task 5.

Contribution

The paper introduces a deep learning approach with state-of-the-art language models for counterfactual detection and extraction, achieving leading performance in the shared task.

Findings

RoBERTa outperformed other models in both subtasks.

First place in extraction accuracy and F1 score.

Second place in counterfactual detection.

Abstract

This paper describes BUT-FIT's submission at SemEval-2020 Task 5: Modelling Causal Reasoning in Language: Detecting Counterfactuals. The challenge focused on detecting whether a given statement contains a counterfactual (Subtask 1) and extracting both antecedent and consequent parts of the counterfactual from the text (Subtask 2). We experimented with various state-of-the-art language representation models (LRMs). We found RoBERTa LRM to perform the best in both subtasks. We achieved the first place in both exact match and F1 for Subtask 2 and ranked second for Subtask 1.