Retrieval Augmented Generation based Large Language Models for Causality Mining

TL;DR

This paper introduces retrieval-augmented generation (RAG) based dynamic prompting schemes to improve large language models' performance in causality detection and mining, addressing limitations of previous unsupervised and supervised methods.

Contribution

The paper proposes novel RAG-based dynamic prompting techniques for LLMs, enhancing causality mining accuracy across multiple datasets and models.

Findings

RAG-based dynamic prompting outperforms static prompting schemes

Significant performance gains across three datasets

Effective for multiple large language models

Abstract

Causality detection and mining are important tasks in information retrieval due to their enormous use in information extraction, and knowledge graph construction. To solve these tasks, in existing literature there exist several solutions -- both unsupervised and supervised. However, the unsupervised methods suffer from poor performance and they often require significant human intervention for causal rule selection, leading to poor generalization across different domains. On the other hand, supervised methods suffer from the lack of large training datasets. Recently, large language models (LLMs) with effective prompt engineering are found to be effective to overcome the issue of unavailability of large training dataset. Yet, in existing literature, there does not exist comprehensive works on causality detection and mining using LLM prompting. In this paper, we present several retrieval-augmented generation (RAG) based dynamic prompting schemes to enhance LLM performance in causality detection and extraction tasks. Extensive experiments over three datasets and five LLMs validate the superiority of our proposed RAG-based dynamic prompting over other static prompting schemes.