Late-to-Early Training: LET LLMs Learn Earlier, So Faster and Better

TL;DR

The paper introduces a Late-to-Early Training (LET) paradigm that leverages late-layer knowledge from pretrained models to accelerate and improve the training of larger language models, reducing computational costs and enhancing performance.

Contribution

It proposes a novel LET approach that guides early training stages using late-layer representations, significantly speeding up training and boosting model capabilities.

Findings

Achieves up to 1.6× training speedup on 1.4B models

Improves downstream task accuracy by nearly 5%

Effective with pretrained models much smaller than the target model

Abstract

As Large Language Models (LLMs) achieve remarkable empirical success through scaling model and data size, pretraining has become increasingly critical yet computationally prohibitive, hindering rapid development. Despite the availability of numerous pretrained LLMs developed at significant computational expense, a fundamental real-world question remains underexplored: \textit{Can we leverage existing small pretrained models to accelerate the training of larger models?} In this paper, we propose a Late-to-Early Training (LET) paradigm that enables LLMs to explicitly learn later knowledge in earlier steps and earlier layers. The core idea is to guide the early layers of an LLM during early training using representations from the late layers of a pretrained (i.e. late training phase) model. We identify two key mechanisms that drive LET's effectiveness: late-to-early-step learning and late-to-early-layer learning. These mechanisms significantly accelerate training convergence while robustly enhancing both language modeling capabilities and downstream task performance, enabling faster training with superior performance. Extensive experiments on 1.4B and 7B parameter models demonstrate LET's efficiency and effectiveness. Notably, when training a 1.4B LLM on the Pile dataset, our method achieves up to 1.6$\times$ speedup with nearly 5\% improvement in downstream task accuracy compared to standard training, even when using a pretrained model with 10$\times$ fewer parameters than the target model.