Method-Based Reasoning for Large Language Models: Extraction, Reuse, and Continuous Improvement

TL;DR

This paper introduces a method-based reasoning system for large language models that extracts, reuses, and continually improves explicit procedures to enhance logical consistency and problem-solving capabilities.

Contribution

It presents a novel approach to augment LLMs with external, reusable methods for reasoning, enabling continual learning and improved performance on complex tasks.

Findings

Improves factual verification in LLM responses

Enhances generalization to complex prompts

Allows user-driven refinement of methods

Abstract

Large language models (LLMs) have shown impressive capabilities across a wide range of language tasks. However, their reasoning process is primarily guided by statistical patterns in training data, which limits their ability to handle novel problems and perform consistent logical reasoning. In this paper, we propose a method-based model that enhances LLMs with explicit, reusable procedures extracted from training content, generated responses, and user interactions. Each method is represented as a pair consisting of a problem and its corresponding solution, stored externally and ranked based on feedback. When a new query is received, the system retrieves and applies the most relevant methods to guide the LLM's response. Our model enables continual learning, method reuse, and logical consistency beyond next-token prediction. Experimental results demonstrate that the system improves factual verification and generalization in complex prompts, and that newly learned methods can outperform earlier ones through user-driven refinement.