Information Re-Organization Improves Reasoning in Large Language Models

TL;DR

This paper introduces InfoRE, a method that reorganizes information by extracting logical relationships and pruning noise before reasoning, significantly enhancing the reasoning capabilities of large language models in contextually aware tasks.

Contribution

The paper proposes a novel information re-organization approach that improves LLM reasoning by emphasizing logical relationship extraction and noise reduction prior to reasoning.

Findings

Achieves an average 4% improvement in reasoning accuracy across multiple tasks.

Effective in zero-shot settings with large models like Llama2-70B, GPT-3.5, and GPT-4.

Enhances understanding and response quality by focusing on logical relationships.

Abstract

Improving the reasoning capabilities of large language models (LLMs) has attracted considerable interest. Recent approaches primarily focus on improving the reasoning process to yield a more precise final answer. However, in scenarios involving contextually aware reasoning, these methods neglect the importance of first identifying logical relationships from the context before proceeding with the reasoning. This oversight could lead to a superficial understanding and interaction with the context, potentially undermining the quality and reliability of the reasoning outcomes. In this paper, we propose an information re-organization (InfoRE) method before proceeding with the reasoning to enhance the reasoning ability of LLMs. Our re-organization method involves initially extracting logical relationships from the contextual content, such as documents or paragraphs, and subsequently pruning redundant content to minimize noise. Then, we utilize the re-organized information in the reasoning process. This enables LLMs to deeply understand the contextual content by clearly perceiving these logical relationships, while also ensuring high-quality responses by eliminating potential noise. To demonstrate the effectiveness of our approach in improving the reasoning ability, we conduct experiments using Llama2-70B, GPT-3.5, and GPT-4 on various contextually aware multi-hop reasoning tasks. Using only a zero-shot setting, our method achieves an average absolute improvement of 4% across all tasks, highlighting its potential to improve the reasoning performance of LLMs. Our source code is available at https://github.com/hustcxx/InfoRE.