Knowledge Navigator: LLM-guided Browsing Framework for Exploratory Search in Scientific Literature

TL;DR

Knowledge Navigator is a system that leverages large language models and clustering techniques to organize scientific literature into a hierarchical structure, facilitating more effective exploratory search and knowledge discovery.

Contribution

The paper introduces a novel LLM-guided framework that structures scientific literature into hierarchical topics, improving exploratory search capabilities.

Findings

Effective organization of literature into hierarchical topics.

Improved search and discovery in scientific domains.

Validated on two new benchmarks, CLUSTREC-COVID and SCITOC.

Abstract

The exponential growth of scientific literature necessitates advanced tools for effective knowledge exploration. We present Knowledge Navigator, a system designed to enhance exploratory search abilities by organizing and structuring the retrieved documents from broad topical queries into a navigable, two-level hierarchy of named and descriptive scientific topics and subtopics. This structured organization provides an overall view of the research themes in a domain, while also enabling iterative search and deeper knowledge discovery within specific subtopics by allowing users to refine their focus and retrieve additional relevant documents. Knowledge Navigator combines LLM capabilities with cluster-based methods to enable an effective browsing method. We demonstrate our approach's effectiveness through automatic and manual evaluations on two novel benchmarks, CLUSTREC-COVID and SCITOC. Our code, prompts, and benchmarks are made publicly available.