SLearnLLM: A Self-Learning Framework for Efficient Domain-Specific Adaptation of Large Language Models

TL;DR

This paper introduces a self-learning framework for domain-specific adaptation of large language models that improves training efficiency by focusing on unknown knowledge within the dataset, reducing training time while maintaining performance.

Contribution

The proposed framework enables LLMs to identify and focus on unknown knowledge in the dataset, significantly reducing training time without sacrificing accuracy.

Findings

Reduces fine-tuning training time in agriculture and medicine domains.

Achieves comparable performance to full dataset fine-tuning.

Enhances training efficiency by filtering out known information.

Abstract

When using supervised fine-tuning (SFT) to adapt large language models (LLMs) to specific domains, a significant challenge arises: should we use the entire SFT dataset for fine-tuning? Common practice often involves fine-tuning directly on the entire dataset due to limited information on the LLM's past training data. However, if the SFT dataset largely overlaps with the model's existing knowledge, the performance gains are minimal, leading to wasted computational resources. Identifying the unknown knowledge within the SFT dataset and using it to fine-tune the model could substantially improve the training efficiency. To address this challenge, we propose a self-learning framework for LLMs inspired by human learning pattern. This framework takes a fine-tuning (SFT) dataset in a specific domain as input. First, the LLMs answer the questions in the SFT dataset. The LLMs then objectively grade the responses and filter out the incorrectly answered QA pairs. Finally, we fine-tune the LLMs based on this filtered QA set. Experimental results in the fields of agriculture and medicine demonstrate that our method substantially reduces training time while achieving comparable improvements to those attained with full dataset fine-tuning. By concentrating on the unknown knowledge within the SFT dataset, our approach enhances the efficiency of fine-tuning LLMs.