A Cross-Domain Evaluation of Approaches for Causal Knowledge Extraction

TL;DR

This paper evaluates different models for extracting causal knowledge from text, showing that span-based approaches with pre-trained embeddings outperform sequence tagging models across multiple domains.

Contribution

It provides a comprehensive comparison of sequence tagging and span-based models for causal extraction, highlighting the effectiveness of span-based methods with BERT embeddings.

Findings

Span-based models outperform sequence tagging models.

Pre-trained embeddings significantly improve causal extraction performance.

Models generalize well across diverse domains.

Abstract

Causal knowledge extraction is the task of extracting relevant causes and effects from text by detecting the causal relation. Although this task is important for language understanding and knowledge discovery, recent works in this domain have largely focused on binary classification of a text segment as causal or non-causal. In this regard, we perform a thorough analysis of three sequence tagging models for causal knowledge extraction and compare it with a span based approach to causality extraction. Our experiments show that embeddings from pre-trained language models (e.g. BERT) provide a significant performance boost on this task compared to previous state-of-the-art models with complex architectures. We observe that span based models perform better than simple sequence tagging models based on BERT across all 4 data sets from diverse domains with different types of cause-effect phrases.