Meta-Reasoning Improves Tool Use in Large Language Models

TL;DR

This paper introduces TECTON, a meta-reasoning system that enhances tool selection in large language models, leading to significant improvements in math reasoning tasks both within and outside the training distribution.

Contribution

The paper proposes a novel two-phase meta-reasoning approach for tool selection, outperforming existing greedy decoding methods in large language models.

Findings

Substantial performance gains on math reasoning datasets

Effective tool selection in out-of-distribution scenarios

Improved reasoning accuracy with meta-reasoning approach

Abstract

External tools help large language models succeed at tasks where they would otherwise typically fail. In existing frameworks, choosing tools at test time relies on naive greedy decoding, regardless of whether the model has been fine-tuned on tool-annotated data or prompted with in-context examples. In contrast, we find that gathering and choosing among a suitable set of candidate tools has greater potential to lead to an optimal selection. We present Tool selECTion via meta-reasONing (TECTON), a two-phase system that first reasons over a task and outputs candidate tools using a custom fine-tuned language modelling head. Then, with the custom head disabled, it meta-reasons (i.e., it reasons over the previous reasoning process) to make a final choice. We show that TECTON results in substantial gains--both in-distribution and out-of-distribution--on a range of math reasoning datasets.