ActiShade: Activating Overshadowed Knowledge to Guide Multi-Hop Reasoning in Large Language Models

TL;DR

ActiShade enhances multi-hop reasoning in large language models by detecting and activating overshadowed knowledge, leading to more accurate retrieval and reasoning through iterative query refinement.

Contribution

This paper introduces ActiShade, a novel method that detects and activates overshadowed knowledge to improve multi-hop reasoning in LLMs, reducing error propagation.

Findings

Outperforms existing methods on multiple datasets

Reduces error accumulation in multi-hop reasoning

Effectively activates overshadowed knowledge during reasoning

Abstract

In multi-hop reasoning, multi-round retrieval-augmented generation (RAG) methods typically rely on LLM-generated content as the retrieval query. However, these approaches are inherently vulnerable to knowledge overshadowing - a phenomenon where critical information is overshadowed during generation. As a result, the LLM-generated content may be incomplete or inaccurate, leading to irrelevant retrieval and causing error accumulation during the iteration process. To address this challenge, we propose ActiShade, which detects and activates overshadowed knowledge to guide large language models (LLMs) in multi-hop reasoning. Specifically, ActiShade iteratively detects the overshadowed keyphrase in the given query, retrieves documents relevant to both the query and the overshadowed keyphrase, and generates a new query based on the retrieved documents to guide the next-round iteration. By supplementing the overshadowed knowledge during the formulation of next-round queries while minimizing the introduction of irrelevant noise, ActiShade reduces the error accumulation caused by knowledge overshadowing. Extensive experiments show that ActiShade outperforms existing methods across multiple datasets and LLMs.