Spectrum: Targeted Training on Signal to Noise Ratio

TL;DR

Spectrum is a novel training method for large language models that selectively fine-tunes modules based on their signal-to-noise ratio, reducing computational resources while maintaining performance.

Contribution

It introduces a SNR-based module selection algorithm for targeted training, enabling efficient fine-tuning with less memory and comparable results to full fine-tuning.

Findings

Spectrum matches full fine-tuning performance

Reduces GPU memory usage significantly

Outperforms existing methods like QLoRA in efficiency

Abstract

Efficiently post-training large language models remains a challenging task due to the vast computational resources required. We present Spectrum, a method that accelerates LLM training by selectively targeting layer modules based on their signal-to-noise ratio (SNR), and freezing the remaining modules. Our approach, which utilizes an algorithm to compute module SNRs prior to training, has shown to effectively match the performance of full fine-tuning while reducing GPU memory usage. Experiments comparing Spectrum to existing methods such as QLoRA demonstrate its effectiveness in terms of model quality and VRAM efficiency in distributed environments.