AGGC: Adaptive Group Gradient Clipping for Stabilizing Large Language Model Training

TL;DR

AGGC introduces an adaptive, group-wise gradient clipping method that stabilizes large language model training by addressing gradient heterogeneity, outperforming traditional methods and enhancing model accuracy and stability.

Contribution

The paper proposes AGGC, a novel adaptive group-wise gradient clipping technique that partitions parameters into functional groups and regulates them based on historical behavior, improving training stability.

Findings

AGGC outperforms LoRA and often surpasses full fine-tuning.

On GSM8K, Mistral-7B with AGGC achieves 72.93% accuracy.

AGGC stabilizes RLVR and enhances logic deduction in LLMs.

Abstract

To stabilize the training of Large Language Models (LLMs), gradient clipping is a nearly ubiquitous heuristic used to alleviate exploding gradients. However, traditional global norm clipping erroneously presupposes gradient homogeneity across different functional modules, leading to an adverse "spill-over" effect where volatile parameters force unnecessary scaling on stable ones. To overcome this, we propose Adaptive Group-wise Gradient Clipping (AGGC). AGGC partitions parameters into groups based on functional types and regulates each according to its historical behavior using an Exponential Moving Average (EMA). Specifically, it constructs an adaptive interval to simultaneously mitigate gradient explosion and vanishing, while employing a time-dependent scheduling mechanism to balance exploration and convergence. Experiments on LLaMA 2-7B, Mistral-7B, and Gemma-7B models show that AGGC consistently outperforms LoRA and frequently surpasses Full Fine-Tuning. On the GSM8K benchmark, Mistral-7B fine-tuned with AGGC achieves an accuracy of 72.93%, exceeding LoRA's 69.5%. AGGC also effectively stabilizes Reinforcement Learning with Verifiable Rewards (RLVR), enhancing the logic deduction of Qwen 2.5 and Llama 3.2 models. Experimental results demonstrate that AGGC effectively addresses the limitations of traditional gradient clipping methods, particularly in overcoming gradient heterogeneity, by utilizing a modular, adaptive clipping strategy to stabilize the training process. Due to its lightweight design, AGGC can be seamlessly integrated into existing post-training pipelines with negligible overhead.