Nonlinear Bipolar Compensation: Handling Outliers in Post-Training Quantization

TL;DR

This paper introduces Nonlinear Bipolar Compensation (NBC), a post-training quantization method that uses nonlinear transformations to better handle outliers, improving accuracy and robustness in model compression.

Contribution

The paper proposes NBC, a novel nonlinear compensation technique with Bipolar Logarithmic Transformation, enhancing outlier handling in post-training quantization.

Findings

NBC improves quantization accuracy across various models.

The method demonstrates robustness against outliers.

Experiments confirm efficiency and generality of NBC.

Abstract

Network quantization has emerged as one of the most practical model compression techniques, which significantly reduces a model's memory and compute consumption by mapping floating-point numbers to low-bit representations. However, existing quantization methods typically suffer from the speed-accuracy tradeoff and limited generalization. To address these issues, recent compensation-based methods offer an efficient yet general solution by introducing additional lightweight linear layers into the quantized network. However, the accuracy of these methods suffers from their limited compensation capability and high sensitivity to outliers. In this paper, we propose Nonlinear Bipolar Compensation (NBC), a post-training quantization approach that introduces nonlinear compensation to reduce the effect of outliers. We further design Bipolar Logarithmic Transformation (BLT), which compresses outliers by mapping both the quantized input and the quantization error into a transformed space. A simple linear layer is then applied for compensation in the transformed space, preserving the efficiency of our method. Extensive experiments across various tasks, models, and quantization methods confirm the effectiveness, efficiency, robustness, and generality of our NBC approach.