Logic Distillation: Learning from Code Function by Function for Decision-making Tasks

TL;DR

This paper introduces Logic Distillation, a framework that enhances small language models' logical reasoning for decision-making by learning from complex functions instantiated by large models, achieving performance comparable to larger models.

Contribution

The paper proposes a novel Logic Distillation framework that enables small language models to acquire logical reasoning capabilities through function-based learning from large models.

Findings

S-LLMs with LD outperform baseline S-LLMs in decision-making tasks.

LD enables S-LLMs to match or surpass L-LLMs in planning performance.

The approach effectively transfers logical reasoning from L-LLMs to S-LLMs.

Abstract

Large language models (LLMs) have garnered increasing attention owing to their powerful logical reasoning capabilities. Generally, larger LLMs (L-LLMs) that require paid interfaces exhibit significantly superior performance compared to smaller LLMs (S-LLMs) that can be deployed on a variety of devices. Knowledge distillation (KD) aims to empower S-LLMs with the capabilities of L-LLMs, while S-LLMs merely mimic the outputs of L-LLMs, failing to get the powerful logical reasoning capabilities. Consequently, S-LLMs are helpless when it comes to planning and decision-making tasks that require logical reasoning capabilities. To tackle the identified challenges, we propose a novel framework called Logic Distillation (LD). Initially, LD employs L-LLMs to instantiate complex instructions into discrete functions and illustrates their usage to establish a function base. Subsequently, based on the function base, LD fine-tunes S-LLMs to learn the logic employed by L-LLMs in planning and decision-making. During testing, LD utilizes a retriever to identify the top-$K$ relevant functions based on instructions and current states, which will be selected and invoked by S-LLMs. Ultimately, S-LLMs yield planning and decision-making outcomes, function by function. Relevant experiments demonstrate that with the assistance of LD, S-LLMs can achieve outstanding results in planning and decision-making tasks, comparable to, or even surpassing, those of L-LLMs.