Self-training Large Language Models through Knowledge Detection

TL;DR

This paper introduces a self-training method for large language models that autonomously labels data and improves generation accuracy, reducing hallucinations and catastrophic forgetting without relying heavily on labeled datasets.

Contribution

It presents a novel reference-free consistency-based self-training framework that enhances LLM performance and robustness in a scalable, cost-effective manner.

Findings

Reduces hallucination in generated outputs

Mitigates catastrophic forgetting in OOD benchmarks

Decreases dependence on labeled datasets

Abstract

Large language models (LLMs) often necessitate extensive labeled datasets and training compute to achieve impressive performance across downstream tasks. This paper explores a self-training paradigm, where the LLM autonomously curates its own labels and selectively trains on unknown data samples identified through a reference-free consistency method. Empirical evaluations demonstrate significant improvements in reducing hallucination in generation across multiple subjects. Furthermore, the selective training framework mitigates catastrophic forgetting in out-of-distribution benchmarks, addressing a critical limitation in training LLMs. Our findings suggest that such an approach can substantially reduce the dependency on large labeled datasets, paving the way for more scalable and cost-effective language model training.