Context-Aware Scientific Knowledge Extraction on Linked Open Data using Large Language Models

TL;DR

This paper presents WISE, a structured, LLM-powered system that efficiently extracts, refines, and synthesizes scientific knowledge from diverse sources, significantly improving recall and depth over traditional methods.

Contribution

Introduces WISE, a novel workflow leveraging large language models and a tree-based architecture for context-aware, non-redundant scientific knowledge extraction from multiple sources.

Findings

WISE reduces processed text by over 80% while increasing recall.

Outputs are more unique and in-depth compared to baselines.

System is adaptable to various scientific domains.

Abstract

The exponential growth of scientific literature challenges researchers extracting and synthesizing knowledge. Traditional search engines return many sources without direct, detailed answers, while general-purpose LLMs may offer concise responses that lack depth or omit current information. LLMs with search capabilities are also limited by context window, yielding short, incomplete answers. This paper introduces WISE (Workflow for Intelligent Scientific Knowledge Extraction), a system addressing these limits by using a structured workflow to extract, refine, and rank query-specific knowledge. WISE uses an LLM-powered, tree-based architecture to refine data, focusing on query-aligned, context-aware, and non-redundant information. Dynamic scoring and ranking prioritize unique contributions from each source, and adaptive stopping criteria minimize processing overhead. WISE delivers detailed, organized answers by systematically exploring and synthesizing knowledge from diverse sources. Experiments on HBB gene-associated diseases demonstrate WISE reduces processed text by over 80% while achieving significantly higher recall over baselines like search engines and other LLM-based approaches. ROUGE and BLEU metrics reveal WISE's output is more unique than other systems, and a novel level-based metric shows it provides more in-depth information. We also explore how the WISE workflow can be adapted for diverse domains like drug discovery, material science, and social science, enabling efficient knowledge extraction and synthesis from unstructured scientific papers and web sources.