Rectifying Belief Space via Unlearning to Harness LLMs' Reasoning

TL;DR

This paper introduces a method to improve LLM reasoning by identifying and unlearning spurious beliefs, leading to more accurate and reliable answers without sacrificing overall performance.

Contribution

The paper presents a novel belief rectification technique using unlearning and explanation-based belief identification to enhance LLM reasoning accuracy.

Findings

Corrects misanswered questions effectively

Improves generalization on unseen data

Maintains overall model performance

Abstract

Large language models (LLMs) can exhibit advanced reasoning yet still generate incorrect answers. We hypothesize that such errors frequently stem from spurious beliefs, propositions the model internally considers true but are incorrect. To address this, we propose a method to rectify the belief space by suppressing these spurious beliefs while simultaneously enhancing true ones, thereby enabling more reliable inferences. Our approach first identifies the beliefs that lead to incorrect or correct answers by prompting the model to generate textual explanations, using our Forward-Backward Beam Search (FBBS). We then apply unlearning to suppress the identified spurious beliefs and enhance the true ones, effectively rectifying the model's belief space. Empirical results on multiple QA datasets and LLMs show that our method corrects previously misanswered questions without harming overall model performance. Furthermore, our approach yields improved generalization on unseen data, suggesting that rectifying a model's belief space is a promising direction for mitigating errors and enhancing overall reliability.