IITK-RSA at SemEval-2020 Task 5: Detecting Counterfactuals

TL;DR

This paper presents ensemble transformer and CNN models for detecting and analyzing counterfactual statements in text, achieving top-10 rankings in SemEval-2020 Task 5.

Contribution

It introduces novel ensemble approaches combining transformer, CNN, and dependency tree features for counterfactual detection and segmentation.

Findings

Achieved 4th and 9th place in leaderboard

Demonstrated effectiveness of transformer-based ensembles

Explored linguistic feature incorporation

Abstract

This paper describes our efforts in tackling Task 5 of SemEval-2020. The task involved detecting a class of textual expressions known as counterfactuals and separating them into their constituent elements. Counterfactual statements describe events that have not or could not have occurred and the possible implications of such events. While counterfactual reasoning is natural for humans, understanding these expressions is difficult for artificial agents due to a variety of linguistic subtleties. Our final submitted approaches were an ensemble of various fine-tuned transformer-based and CNN-based models for the first subtask and a transformer model with dependency tree information for the second subtask. We ranked 4-th and 9-th in the overall leaderboard. We also explored various other approaches that involved the use of classical methods, other neural architectures and the incorporation of different linguistic features.