RAISE: Enhancing Scientific Reasoning in LLMs via Step-by-Step Retrieval

TL;DR

RAISE is a novel framework that improves scientific reasoning in large language models by incorporating step-by-step retrieval of relevant documents, leading to better logical relevance and domain knowledge integration.

Contribution

It introduces a three-step retrieval-augmented approach for scientific reasoning, combining problem decomposition, logical query generation, and targeted document retrieval.

Findings

RAISE outperforms baseline models on scientific reasoning benchmarks.

It retrieves documents that are both domain-relevant and logically pertinent.

The approach enhances reasoning accuracy by integrating logical relevance in retrieval.

Abstract

Scientific reasoning requires not only long-chain reasoning processes, but also knowledge of domain-specific terminologies and adaptation to updated findings. To deal with these challenges for scientific reasoning, we introduce RAISE, a step-by-step retrieval-augmented framework which retrieves logically relevant documents from in-the-wild corpus. RAISE is divided into three steps: problem decomposition, logical query generation, and logical retrieval. We observe that RAISE consistently outperforms other baselines on scientific reasoning benchmarks. We analyze that unlike other baselines, RAISE retrieves documents that are not only similar in terms of the domain knowledge, but also documents logically more relevant.