SOAR: Scale Optimization for Accurate Reconstruction in NVFP4 Quantization

TL;DR

SOAR is a post-training quantization framework that optimizes scales in NVFP4 format, significantly improving LLM accuracy without hardware changes.

Contribution

It introduces joint and decoupled scale optimization techniques, enhancing NVFP4 quantization performance over existing methods.

Findings

Outperforms existing NVFP4 quantization baselines

Achieves higher accuracy at the same memory footprint

No additional hardware overhead required

Abstract

NVFP4 has recently emerged as an efficient 4-bit microscaling format for large language models (LLMs), offering superior numerical fidelity with native hardware support. However, existing methods often yield suboptimal performance due to inflexible scale selection and the coupled treatment of quantization and dequantization scales. To address these issues, we propose Scale Optimization for Accurate Reconstruction (SOAR), a novel post-training quantization framework that improves the accuracy of NVFP4 quantization. At its core, SOAR features Closed-form Joint Scale Optimization (CJSO), which jointly optimizes global and block-wise scales via analytical solutions derived from reconstruction error minimization. Furthermore, it incorporates Decoupled Scale Search (DSS). DSS decouples the high-precision quantization scale from its constrained dequantization counterpart, and performs discrete search to mitigate precision loss from scale quantization. Extensive experiments across multiple LLMs show that our method consistently outperforms existing NVFP4 quantization baselines, achieving superior accuracy under the same memory footprint with no additional hardware overhead. The code and models will be available at https://github.com/steven-bao1/SOAR.