SiLQ: Simple Large Language Model Quantization-Aware Training

TL;DR

SiLQ introduces a straightforward quantization-aware training method for large language models that significantly improves accuracy with minimal additional training cost, compatible with various architectures and deployment scenarios.

Contribution

The paper presents a simple, end-to-end quantization-aware training approach that outperforms existing methods with minimal training overhead and broad applicability.

Findings

Outperforms leading quantization methods on multiple benchmarks.

Requires less than 0.1% additional training budget.

Compatible with various model architectures and deployment setups.

Abstract

Large language models can be quantized to reduce inference time latency, model size, and energy consumption, thereby delivering a better user experience at lower cost. A challenge exists to deliver quantized models with minimal loss of accuracy in reasonable time, and in particular to do so without requiring mechanisms incompatible with specialized inference accelerators. Here, we demonstrate a simple, end-to-end quantization-aware training approach that, with an increase in total model training budget of less than 0.1%, outperforms the leading published quantization methods by large margins on several modern benchmarks, with both base and instruct model variants. The approach easily generalizes across different model architectures, can be applied to activations, cache, and weights, and requires the introduction of no additional operations to the model other than the quantization itself.