Reference Trustable Decoding: A Training-Free Augmentation Paradigm for Large Language Models

TL;DR

Reference Trustable Decoding (RTD) is a training-free method that enhances large language models' adaptability to downstream tasks by constructing a reference datastore and optimizing output distributions, offering trustworthiness and low inference costs.

Contribution

RTD introduces a novel, training-free paradigm for LLM augmentation that constructs a reference datastore and improves response trustworthiness without fine-tuning.

Findings

RTD achieves competitive performance on various benchmarks.

RTD operates with low inference costs and high efficiency.

RTD is orthogonal and compatible with traditional fine-tuning methods.

Abstract

Large language models (LLMs) have rapidly advanced and demonstrated impressive capabilities. In-Context Learning (ICL) and Parameter-Efficient Fine-Tuning (PEFT) are currently two mainstream methods for augmenting LLMs to downstream tasks. ICL typically constructs a few-shot learning scenario, either manually or by setting up a Retrieval-Augmented Generation (RAG) system, helping models quickly grasp domain knowledge or question-answering patterns without changing model parameters. However, this approach involves trade-offs, such as slower inference speed and increased space occupancy. PEFT assists the model in adapting to tasks through minimal parameter modifications, but the training process still demands high hardware requirements, even with a small number of parameters involved. To address these challenges, we propose Reference Trustable Decoding (RTD), a paradigm that allows models to quickly adapt to new tasks without fine-tuning, maintaining low inference costs. RTD constructs a reference datastore from the provided training examples and optimizes the LLM's final vocabulary distribution by flexibly selecting suitable references based on the input, resulting in more trustable responses and enabling the model to adapt to downstream tasks at a low cost. Experimental evaluations on various LLMs using different benchmarks demonstrate that RTD establishes a new paradigm for augmenting models to downstream tasks. Furthermore, our method exhibits strong orthogonality with traditional methods, allowing for concurrent usage. Our code can be found at https://github.com/ShiLuohe/ReferenceTrustableDecoding