TAIA: Large Language Models are Out-of-Distribution Data Learners

TL;DR

This paper introduces a novel inference-time method, TAIA, that improves large language models' performance in data-scarce, domain-mismatched scenarios by selectively fine-tuning attention parameters and inferring with only those parameters.

Contribution

The paper reveals that only attention parameters benefit from fine-tuning in mismatched domains and proposes TAIA, a method that enhances LLM performance by training all parameters but inferring with only attention.

Findings

TAIA outperforms fully fine-tuned and base models across multiple tasks.

Selective attention parameter tuning improves robustness to data mismatch.

TAIA enhances task-specific performance and resists jailbreaking tuning.

Abstract

Fine-tuning on task-specific question-answer pairs is a predominant method for enhancing the performance of instruction-tuned large language models (LLMs) on downstream tasks. However, in certain specialized domains, such as healthcare or harmless content generation, it is nearly impossible to obtain a large volume of high-quality data that matches the downstream distribution. To improve the performance of LLMs in data-scarce domains with domain-mismatched data, we re-evaluated the Transformer architecture and discovered that not all parameter updates during fine-tuning contribute positively to downstream performance. Our analysis reveals that within the self-attention and feed-forward networks, only the fine-tuned attention parameters are particularly beneficial when the training set's distribution does not fully align with the test set. Based on this insight, we propose an effective inference-time intervention method: Training All parameters but Inferring with only Attention (\trainallInfAttn). We empirically validate \trainallInfAttn using two general instruction-tuning datasets and evaluate it on seven downstream tasks involving math, reasoning, and knowledge understanding across LLMs of different parameter sizes and fine-tuning techniques. Our comprehensive experiments demonstrate that \trainallInfAttn achieves superior improvements compared to both the fully fine-tuned model and the base model in most scenarios, with significant performance gains. The high tolerance of \trainallInfAttn to data mismatches makes it resistant to jailbreaking tuning and enhances specialized tasks using general data. Code is available in \url{https://github.com/pixas/TAIA_LLM}.