Detecting Miscitation on the Scholarly Web through LLM-Augmented Text-Rich Graph Learning

TL;DR

This paper presents LAGMiD, a novel framework combining large language models and graph neural networks to detect miscitations in scholarly citations with high accuracy and efficiency.

Contribution

The work introduces an evidence-chain reasoning mechanism and a knowledge distillation approach to enhance miscitation detection using LLMs and GNNs.

Findings

Achieves state-of-the-art accuracy in miscitation detection

Reduces inference costs significantly compared to baseline methods

Effective multi-hop citation tracing through chain-of-thought prompting

Abstract

Scholarly web is a vast network of knowledge connected by citations. However, this system is increasingly compromised by miscitation, where references do not support or even contradict the claims they are cited for. Current miscitation detection methods, which primarily rely on semantic similarity or network anomalies, struggle to capture the nuanced relationship between a citation's context and its place in the wider network. While large language models (LLMs) offer powerful capabilities in semantic reasoning for this task, their deployment is hindered by hallucination risks and high computational costs. In this work, we introduce LLM-Augmented Graph Learning-based Miscitation Detector (LAGMiD), a novel framework that leverages LLMs for deep semantic reasoning over citation graphs and distills this knowledge into graph neural networks (GNNs) for efficient and scalable miscitation detection. Specifically, LAGMiD introduces an evidence-chain reasoning mechanism, which uses chain-of-thought prompting, to perform multi-hop citation tracing and assess semantic fidelity. To reduce LLM inference costs, we design a knowledge distillation method aligning GNN embeddings with intermediate LLM reasoning states. A collaborative learning strategy further routes complex cases to the LLM while optimizing the GNN for structure-based generalization. Experiments on three real-world benchmarks show that LAGMiD achieves state-of-the-art miscitation detection with significantly reduced inference cost.