Language Models Coupled with Metacognition Can Outperform Reasoning Models

TL;DR

This paper introduces SOFAI-LM, a hybrid approach combining large language models with metacognitive feedback and reasoning modules, significantly improving reasoning performance and efficiency across diverse tasks.

Contribution

The paper generalizes the SOFAI architecture to coordinate LLMs with LRMs via metacognition, enhancing reasoning capabilities without additional fine-tuning.

Findings

SOFAI-LM improves reasoning accuracy to match or surpass LRMs.

The approach reduces inference time compared to standalone LRMs.

It effectively adapts to different problem domains like graph coloring and code debugging.

Abstract

Large language models (LLMs) excel in speed and adaptability across various reasoning tasks, but they often struggle when strict logic or constraint enforcement is required. In contrast, Large Reasoning Models (LRMs) are specifically designed for complex, step-by-step reasoning, although they come with significant computational costs and slower inference times. To address these trade-offs, we employ and generalize the SOFAI (Slow and Fast AI) cognitive architecture into SOFAI-LM, which coordinates a fast LLM with a slower but more powerful LRM through metacognition. The metacognitive module actively monitors the LLM's performance and provides targeted, iterative feedback with relevant examples. This enables the LLM to progressively refine its solutions without requiring the need for additional model fine-tuning. Extensive experiments on graph coloring and code debugging problems demonstrate that our feedback-driven approach significantly enhances the problem-solving capabilities of the LLM. In many instances, it achieves performance levels that match or even exceed those of standalone LRMs while requiring considerably less time. Additionally, when the LLM and feedback mechanism alone are insufficient, we engage the LRM by providing appropriate information collected during the LLM's feedback loop, tailored to the specific characteristics of the problem domain and leads to improved overall performance. Evaluations on two contrasting domains: graph coloring, requiring globally consistent solutions, and code debugging, demanding localized fixes, demonstrate that SOFAI-LM enables LLMs to match or outperform standalone LRMs in accuracy while maintaining significantly lower inference time.