Mix-CPT: A Domain Adaptation Framework via Decoupling Knowledge Learning and Format Alignment

TL;DR

Mix-CPT is a novel domain adaptation framework for large language models that decouples knowledge learning from format alignment, enabling efficient adaptation with limited data and improved task performance across domains.

Contribution

The paper introduces Mix-CPT, a framework that separates knowledge learning and format alignment, incorporating knowledge mixture pre-training and self-distillation to enhance domain adaptation of LLMs.

Findings

Improves task-solving capabilities on target and general domains

Efficient adaptation with limited training samples

Outperforms traditional domain adaptation methods

Abstract

Adapting general large language models (LLMs) to specialized domains presents great challenges due to varied data distributions. This adaptation typically requires continual pre-training on massive domain-specific corpora to facilitate knowledge memorization, followed by training to apply this knowledge following human instructions and preferences. However, this method may result in inefficient knowledge memorization due to a lack of awareness of knowledge utilization and imposes substantial demands on LLMs to simultaneously learn knowledge utilization and format alignment with limited training samples. To facilitate the domain adaptation of LLM, we revise this process and propose a new domain adaptation framework including domain knowledge learning and general format alignment, called Mix-CPT. Specifically, we first conduct a knowledge mixture continual pre-training that concurrently focuses on knowledge memorization and utilization, allowing for mutual reinforcement. To avoid catastrophic forgetting during the continual pre-training process, we further incorporate a logit swap self-distillation constraint. Subsequently, leveraging the knowledge and capabilities acquired during continual pre-training, we efficiently perform instruction tuning and alignment with a few general training samples to achieve format alignment. Extensive experiments demonstrate that our proposed Mix-CPT framework can simultaneously improve the task-solving capabilities of LLMs on the target and general domains compared to the traditional adaptation methods.