Shuttle Between the Instructions and the Parameters of Large Language Models

TL;DR

This paper introduces SHIP, a neural network framework that models the relationship between instructions and parameters in large language models, enabling better instruction deduction and induction.

Contribution

The paper proposes a novel framework, SHIP, that learns mutual mappings between instructions and parameters, enhancing LLMs' reasoning abilities beyond existing methods.

Findings

SHIP outperforms baselines in instruction deduction

SHIP surpasses baselines in inductive reasoning

Effective combination of mappings improves reasoning

Abstract

The interaction with Large Language Models (LLMs) through instructions has been extensively investigated in the research community. While instructions have been widely used as the guidelines for task solving, this paper further notices that both instructions and parameters are the compression of task data. Therefore, they could be strongly correlated and can be learned to predict one from the other. This paper proposes a novel neural network framework, SHIP (\textbf{Sh}uttle between the \textbf{I}nstructions and the \textbf{P}arameters), to model and learn the mutual mappings between the instructions and the parameters of LLMs. We verify that SHIP can effectively map one of the instructions/parameters to the other by evaluating it on the tasks of instruction deduction and induction. The results show that SHIP performs better than existing baseline methods in terms of deductive capabilities while significantly surpassing them in inductive capabilities. Moreover, SHIP can effectively combine the two mapping processes to perform excellent inductive reasoning. The code and data for this paper are released at https://anonymous.4open.science/r/Shuttle-Between-Instructions-Parameters/.