PRAGyan -- Connecting the Dots in Tweets

TL;DR

This paper presents PRAGyan, a method combining Knowledge Graphs and Large Language Models to improve causal analysis of tweets, enhancing interpretability and accuracy over baseline models like GPT-3.5 Turbo.

Contribution

It introduces a retrieval-augmented generation approach using KGs with LLMs for deeper causal analysis in social media data.

Findings

KG-enhanced LLM outperforms baseline GPT-3.5 Turbo in causal reasoning.

Qualitative analysis shows improved interpretability and insights.

Quantitative metrics indicate a 10% performance improvement.

Abstract

As social media platforms grow, understanding the underlying reasons behind events and statements becomes crucial for businesses, policymakers, and researchers. This research explores the integration of Knowledge Graphs (KGs) with Large Language Models (LLMs) to perform causal analysis of tweets dataset. The LLM aided analysis techniques often lack depth in uncovering the causes driving observed effects. By leveraging KGs and LLMs, which encode rich semantic relationships and temporal information, this study aims to uncover the complex interplay of factors influencing causal dynamics and compare the results obtained using GPT-3.5 Turbo. We employ a Retrieval-Augmented Generation (RAG) model, utilizing a KG stored in a Neo4j (a.k.a PRAGyan) data format, to retrieve relevant context for causal reasoning. Our approach demonstrates that the KG-enhanced LLM RAG can provide improved results when compared to the baseline LLM (GPT-3.5 Turbo) model as the source corpus increases in size. Our qualitative analysis highlights the advantages of combining KGs with LLMs for improved interpretability and actionable insights, facilitating informed decision-making across various domains. Whereas, quantitative analysis using metrics such as BLEU and cosine similarity show that our approach outperforms the baseline by 10\%.