Extracting Cause-Effect Pairs from a Sentence with a Dependency-Aware Transformer Model

TL;DR

This paper introduces DepBERT, a novel transformer-based model that incorporates dependency trees to improve cause-effect phrase extraction from sentences, outperforming existing supervised methods across multiple datasets.

Contribution

DepBERT is the first model to integrate dependency tree information directly into a transformer framework for causality extraction.

Findings

DepBERT outperforms state-of-the-art supervised causality extraction methods.

Incorporating dependency trees improves extraction accuracy.

The model performs well across three different datasets.

Abstract

Extracting cause and effect phrases from a sentence is an important NLP task, with numerous applications in various domains, including legal, medical, education, and scientific research. There are many unsupervised and supervised methods proposed for solving this task. Among these, unsupervised methods utilize various linguistic tools, including syntactic patterns, dependency tree, dependency relations, etc. among different sentential units for extracting the cause and effect phrases. On the other hand, the contemporary supervised methods use various deep learning based mask language models equipped with a token classification layer for extracting cause and effect phrases. Linguistic tools, specifically, dependency tree, which organizes a sentence into different semantic units have been shown to be very effective for extracting semantic pairs from a sentence, but existing supervised methods do not have any provision for utilizing such tools within their model framework. In this work, we propose DepBERT, which extends a transformer-based model by incorporating dependency tree of a sentence within the model framework. Extensive experiments over three datasets show that DepBERT is better than various state-of-the art supervised causality extraction methods.