Improving Scientific Document Retrieval with Academic Concept Index

TL;DR

This paper introduces an academic concept index to improve scientific document retrieval by enhancing query generation and context augmentation, leading to better alignment with scientific concepts and improved retrieval accuracy.

Contribution

It proposes a structured academic concept index and two novel methods, CCQGen and CCExpand, to enhance retrieval by leveraging concept coverage and focus.

Findings

Improved retrieval performance with concept-aware methods

Enhanced query diversity and coverage

Better alignment with scientific concepts

Abstract

Adapting general-domain retrievers to scientific domains is challenging due to the scarcity of large-scale domain-specific relevance annotations and the substantial mismatch in vocabulary and information needs. Recent approaches address these issues through two independent directions that leverage large language models (LLMs): (1) generating synthetic queries for fine-tuning, and (2) generating auxiliary contexts to support relevance matching. However, both directions overlook the diverse academic concepts embedded within scientific documents, often producing redundant or conceptually narrow queries and contexts. To address this limitation, we introduce an academic concept index, which extracts key concepts from papers and organizes them guided by an academic taxonomy. This structured index serves as a foundation for improving both directions. First, we enhance the synthetic query generation with concept coverage-based generation (CCQGen), which adaptively conditions LLMs on uncovered concepts to generate complementary queries with broader concept coverage. Second, we strengthen the context augmentation with concept-focused auxiliary contexts (CCExpand), which leverages a set of document snippets that serve as concise responses to the concept-aware CCQGen queries. Extensive experiments show that incorporating the academic concept index into both query generation and context augmentation leads to higher-quality queries, better conceptual alignment, and improved retrieval performance.