A Training-Free Regeneration Paradigm: Contrastive Reflection Memory Guided Self-Verification and Self-Improvement

TL;DR

This paper introduces a training-free regeneration approach for large language models that uses a contrastive reflection memory to guide self-verification and regeneration, improving accuracy efficiently without iterative correction.

Contribution

The proposed method is a training-free paradigm that leverages contrastive reflection memory for efficient self-verification and regeneration, avoiding costly iterative processes.

Findings

Outperforms prior methods on nine diverse benchmarks.

Maintains low computational cost compared to existing approaches.

Effective across various tasks and model scales.

Abstract

Verification-guided self-improvement has recently emerged as a promising approach to improving the accuracy of large language model (LLM) outputs. However, existing approaches face a trade-off between inference efficiency and accuracy: iterative verification-rectification is computationally expensive and prone to being trapped in faulty reasoning, while best-of-N selection requires extensive sampling without addressing internal model flaws. We propose a training-free regeneration paradigm that leverages an offline-curated contrastive Reflection Memory (RM) to provide corrective guidance, while regenerating from scratch helps break out of faulty reasoning. At inference time, the method performs RM-guided self-verification followed by a single RM-guided regeneration, avoiding both iterative correction and multi-sample selection. We evaluated our method on nine benchmarks that span algorithmic, reasoning, symbolic, and domain-specific tasks in both small- and large-scale LLMs. Experiment results show that our method outperforms prior methods while maintaining low computational cost.