Yseop at SemEval-2020 Task 5: Cascaded BERT Language Model for Counterfactual Statement Analysis

TL;DR

This paper presents a hybrid BERT-based system for detecting and evaluating counterfactual sentences, achieving high F1 scores in SemEval-2020 Task 5 by combining classification and sequence identification techniques.

Contribution

The authors develop a cascaded BERT and MLP system that effectively handles counterfactual sentence detection and sequence delimitation, outperforming similar complex models.

Findings

Achieved F1 score of 85.00% in Task 1

Achieved F1 score of 83.90% in Task 2

Using syntactic and semantic features as cascaded inputs improves sequence delimitation

Abstract

In this paper, we explore strategies to detect and evaluate counterfactual sentences. We describe our system for SemEval-2020 Task 5: Modeling Causal Reasoning in Language: Detecting Counterfactuals. We use a BERT base model for the classification task and build a hybrid BERT Multi-Layer Perceptron system to handle the sequence identification task. Our experiments show that while introducing syntactic and semantic features does little in improving the system in the classification task, using these types of features as cascaded linear inputs to fine-tune the sequence-delimiting ability of the model ensures it outperforms other similar-purpose complex systems like BiLSTM-CRF in the second task. Our system achieves an F1 score of 85.00% in Task 1 and 83.90% in Task 2.