SituatedThinker: Grounding LLM Reasoning with Real-World through Situated Thinking

TL;DR

SituatedThinker enhances large language models by grounding their reasoning in real-world contexts through situated thinking, combining internal knowledge with external information via reinforcement learning to improve performance on diverse reasoning tasks.

Contribution

Introduces SituatedThinker, a framework that grounds LLM reasoning in real-world contexts using reinforcement learning and external interfaces, surpassing knowledge limitations.

Findings

Significant improvements on multi-hop question-answering and mathematical reasoning benchmarks.

Strong performance on unseen tasks like KBQA, TableQA, and text-based games.

Demonstrates generalizable real-world grounded reasoning capability.

Abstract

Recent advances in large language models (LLMs) demonstrate their impressive reasoning capabilities. However, the reasoning confined to internal parametric space limits LLMs' access to real-time information and understanding of the physical world. To overcome this constraint, we introduce SituatedThinker, a novel framework that enables LLMs to ground their reasoning in real-world contexts through situated thinking, which adaptively combines both internal knowledge and external information with predefined interfaces. By utilizing reinforcement learning, SituatedThinker incentivizes deliberate reasoning with the real world to acquire information and feedback, allowing LLMs to surpass their knowledge boundaries and enhance reasoning. Experimental results demonstrate significant performance improvements on multi-hop question-answering and mathematical reasoning benchmarks. Furthermore, SituatedThinker demonstrates strong performance on unseen tasks, such as KBQA, TableQA, and text-based games, showcasing the generalizable real-world grounded reasoning capability. Our codes are available at https://github.com/jnanliu/SituatedThinker.