Post-Training Quantization for Vision Mamba with k-Scaled Quantization and Reparameterization

TL;DR

This paper introduces a novel post-training quantization method for Vision Mamba, a vision model based on structured state-space models, achieving minimal accuracy loss and enabling efficient deployment.

Contribution

It proposes three core techniques—k-scaled token-wise quantization, reparameterization, and factorization—to effectively quantize Vision Mamba without significant accuracy degradation.

Findings

Achieves only 0.8-1.2% accuracy loss on ImageNet-1k after quantization.

Introduces a reparameterization technique to simplify hidden state quantization.

Reduces computational overhead through a factor-determining method.

Abstract

The Mamba model, utilizing a structured state-space model (SSM), offers linear time complexity and demonstrates significant potential. Vision Mamba (ViM) extends this framework to vision tasks by incorporating a bidirectional SSM and patch embedding, surpassing Transformer-based models in performance. While model quantization is essential for efficient computing, existing works have focused solely on the original Mamba model and have not been applied to ViM. Additionally, they neglect quantizing the SSM layer, which is central to Mamba and can lead to substantial error propagation by naive quantization due to its inherent structure. In this paper, we focus on the post-training quantization (PTQ) of ViM. We address the issues with three core techniques: 1) a k-scaled token-wise quantization method for linear and convolutional layers, 2) a reparameterization technique to simplify hidden state quantization, and 3) a factor-determining method that reduces computational overhead by integrating operations. Through these methods, the error caused by PTQ can be mitigated. Experimental results on ImageNet-1k demonstrate only a 0.8-1.2\% accuracy degradation due to PTQ, highlighting the effectiveness of our approach.