The Fourth State: Signed-Zero Ternary for Stable LLM Quantization (and More)

TL;DR

This paper introduces Signed-Zero Ternary (SZT), a 2-bit quantization method that enhances gradient information without performance loss, potentially improving information density in large language model quantization.

Contribution

The paper presents SZT, a novel 2-bit quantization scheme that offers deterministic gradient information without forward-path penalties, improving upon existing quantization methods.

Findings

SZT provides gradient information with no forward-path penalty.

SZT may improve information density compared to non-quantized models.

The method is applicable to stable LLM quantization.

Abstract

Quantization is usually regarded as a means to trade quality of performance for reduced compute requirements, i.e., as a suboptimal approximation. However, if examined in terms of a fixed overall resource budget, a very different perspective arises. We introduce Signed-Zero Ternary (SZT), a 2-bit quantization that deterministically provides gradient information with no forward-path penalty. Our analysis provides evidence that it may improve information density compared to non-quantized alternatives.