To FP8 and Back Again: Quantifying Reduced Precision Effects on LLM Training Stability

TL;DR

This paper investigates the stability and robustness of FP8 reduced-precision training for large language models, highlighting current limitations and proposing new evaluation methods to guide future research.

Contribution

It introduces new evaluation techniques and a metric for loss landscape sharpness, analyzing the impact of reduced precision on LLM training stability.

Findings

FP8 training methods are not yet robust enough for cost-effective use

Reduced precision affects training stability across seeds, learning rates, and datasets

Simulation of bit reductions reveals the relationship between precision and stability

Abstract

The massive computational costs associated with large language model (LLM) pretraining have spurred great interest in reduced-precision floating-point representations to accelerate the process. As a result, the BrainFloat16 (BF16) precision has become the de facto standard for LLM training, with hardware support included in recent generations of accelerators. This trend has gone even further in the latest processors, where FP8 has recently been introduced. However, prior experience with FP16, which was found to be less stable than BF16, raises concerns as to whether FP8, with even fewer bits than FP16, can be a cost-effective option for LLM training. We argue that reduced-precision training schemes must have similar training stability and hyperparameter sensitivities to their higher-precision counterparts in order to be cost-effective. However, we find that currently available methods for FP8 training are not robust enough to allow their use as economical replacements. This prompts us to investigate the stability of reduced-precision LLM training in terms of robustness across random seeds, learning rates, and datasets. To this end, we propose new evaluation techniques and a new metric for quantifying loss landscape sharpness in autoregressive language models. By simulating incremental bit reductions in floating-point representations, we analyze the relationship between representational power and training stability with the intent of aiding future research into the field.