Evolution without Large Models: Training Language Model with Task Principles

TL;DR

This paper introduces a self-evolution approach for training language models that leverages task principles to generate training data, reducing reliance on large models and lowering carbon emissions.

Contribution

The paper presents a novel method combining multi-level principle generation and principle-based data creation to improve language model training efficiency and sustainability.

Findings

Significant performance improvements over baseline methods

Reduced carbon emissions due to limited use of large models

Effective data generation using task principles

Abstract

A common training approach for language models involves using a large-scale language model to expand a human-provided dataset, which is subsequently used for model training.This method significantly reduces training costs by eliminating the need for extensive human data annotation. However, it still faces challenges such as high carbon emissions during data augmentation and the risk of data leakage when we use closed-source LLMs. To address these issues, we propose a self-evolution method for language models. First, we introduce the Multi-level Principle Generation, which enables a large-scale model to summarize task-completion principles based on a small amount of task data. Then, we propose the Principle-based Instance Generation, in which a smaller-scale language model uses these task principles to generate a large amount of data. This data is then used for model training. Experimental results show that our proposed method significantly improves model performance compared to directly using a smaller-scale language model to generate data. Additionally, since we only use the large-scale language model to generate the task-completion principles, the carbon emissions associated with training the model are greatly reduced.