Distilling Tool Knowledge into Language Models via Back-Translated Traces

TL;DR

This paper introduces a method to distill external tool reasoning into language models using natural language traces, improving math problem-solving without external tools during inference.

Contribution

It presents a novel pipeline that converts tool-based reasoning traces into natural language, enabling models to internalize reasoning patterns through fine-tuning.

Findings

Enhanced performance on math benchmarks without external tools

Effective conversion of tool traces into natural language explanations

Small models can internalize complex reasoning patterns

Abstract

Large language models (LLMs) often struggle with mathematical problems that require exact computation or multi-step algebraic reasoning. Tool-integrated reasoning (TIR) offers a promising solution by leveraging external tools such as code interpreters to ensure correctness, but it introduces inference-time dependencies that hinder scalability and deployment. In this work, we propose a new paradigm for distilling tool knowledge into LLMs purely through natural language. We first construct a Solver Agent that solves math problems by interleaving planning, symbolic tool calls, and reflective reasoning. Then, using a back-translation pipeline powered by multiple LLM-based agents, we convert interleaved TIR traces into natural language reasoning traces. A Translator Agent generates explanations for individual tool calls, while a Rephrase Agent merges them into a fluent and globally coherent narrative. Empirically, we show that fine-tuning a small open-source model on these synthesized traces enables it to internalize both tool knowledge and structured reasoning patterns, yielding gains on competition-level math benchmarks without requiring tool access at inference.