Efficient Task Adaptation in Large Language Models via Selective Parameter Optimization

TL;DR

This paper introduces a selective parameter fine-tuning method for large language models that preserves core knowledge while adapting to specific domains, improving transferability and reducing forgetting.

Contribution

It proposes a novel importance evaluation method to distinguish core and non-core parameters, enabling targeted fine-tuning and better domain adaptation.

Findings

Mitigates catastrophic forgetting in LLM fine-tuning

Enhances model adaptability to scientific, medical, and physical tasks

Shows improved performance on GPT-J and LLaMA-3 models

Abstract

Large Language Models (LLMs) have demonstrated excellent performance in general language understanding, generation and other tasks. However, when fine-tuning for specific domain tasks, the general knowledge accumulated in the pre-training phase is often partially overwritten or forgotten due to parameter updates, which severely limits the generalization ability and transferability of LLMs. Traditional fine-tuning strategies mostly train on the entire parameter space, ignoring the heterogeneity of model parameters, that is, some parameters are extremely important for general tasks, while other parameters are more sensitive to specific tasks. To alleviate the above problems, this paper innovatively proposes a parameter element importance evaluation method, which divides parameters into "core parameters" and "non-core parameters" by distinguishing the importance of parameters for general language ability tasks and specific domain tasks, and fixes the core parameters during fine-tuning, and only fine-tunes the non-core parameters. Extensive experiments on scientific, medical and physical tasks using GPT-J and LLaMA-3 show that our method can mitigate catastrophic forgetting while enhancing the adaptability of the model.