BATQuant: Outlier-resilient MXFP4 Quantization via Learnable Block-wise Optimization

TL;DR

BATQuant is a novel quantization method for MXFP4 formats that improves outlier resilience and performance in large language models by restricting transformations and optimizing distribution shaping.

Contribution

It introduces BATQuant, a block-wise affine transformation approach with Kronecker decomposition and learnable clipping, addressing format mismatch issues in MXFP4 quantization.

Findings

Achieves up to 96.43% of full-precision performance on multimodal benchmarks.

Outperforms existing quantization methods across various tasks.

Establishes new state-of-the-art results under W4A4KV16 configurations.

Abstract

Microscaling floating-point (MXFP) formats have emerged as a promising standard for deploying Multi-modal Large Language Models (MLLMs) and Large Language Models (LLMs) on modern accelerator architectures. However, existing Post-Training Quantization (PTQ) methods, particularly rotation-based techniques designed for integer formats, suffer from severe performance collapse when applied to MXFP4. Recent studies attribute this failure to a fundamental format mismatch: global orthogonal rotations inadvertently transfer outlier energy across quantization blocks, inducing new outliers that disrupt local block-wise scaling, while often creating bimodal activation distributions that underutilize the limited quantization range. To address these issues, we propose BATQuant (Block-wise Affine Transformation), which restricts transformations to align with MXFP granularity to prevent cross-block outlier propagation, while relaxing orthogonality constraints to optimize distribution shaping. To ensure parameter efficiency, we introduce Global and Private Kronecker (GPK) decomposition to effectively reduces storage and runtime overhead and incorporate Block-wise Learnable Clipping to suppress residual outliers. Extensive experiments on both MLLMs and LLMs demonstrate that BATQuant establishes new state-of-the-art results under aggressive W4A4KV16 configurations, recovering up to 96.43% of full-precision performance on multimodal benchmarks and clearly outperforming existing methods across diverse tasks.