LLM Guided Inductive Inference for Solving Compositional Problems

TL;DR

This paper introduces REBEL, a recursive reasoning framework that enhances large language models' ability to solve complex, open-world problems by decomposing tasks and utilizing external tools through natural language instructions.

Contribution

REBEL is a novel recursive reasoning method enabling LLMs to perform deep, compositional problem-solving with external tools in an open-world setting.

Findings

REBEL improves reasoning over complex, nested problems.

It effectively integrates external tools via natural language.

REBEL demonstrates capabilities in conversational, compositional tasks.

Abstract

While large language models (LLMs) have demonstrated impressive performance in question-answering tasks, their performance is limited when the questions require knowledge that is not included in the model's training data and can only be acquired through direct observation or interaction with the real world. Existing methods decompose reasoning tasks through the use of modules invoked sequentially, limiting their ability to answer deep reasoning tasks. We introduce a method, Recursion based extensible LLM (REBEL), which handles open-world, deep reasoning tasks by employing automated reasoning techniques like dynamic planning and forward-chaining strategies. REBEL allows LLMs to reason via recursive problem decomposition and utilization of external tools. The tools that REBEL uses are specified only by natural language description. We further demonstrate REBEL capabilities on a set of problems that require a deeply nested use of external tools in a compositional and conversational setting.