Beacon: Post-Training Quantization with Integrated Grid Selection

TL;DR

Beacon introduces a straightforward, tuning-free post-training quantization method that automatically determines optimal scaling factors, enabling efficient and competitive model compression without extensive calibration or heuristic tuning.

Contribution

It presents a novel algorithm for per-channel PTQ that eliminates manual tuning by leveraging the geometry of scalar quantization, simplifying the quantization process.

Findings

Achieves competitive accuracy with state-of-the-art methods

Does not require back-propagation or large calibration sets

Simplifies post-training quantization process

Abstract

Quantization is a widely used compression technique for reducing the memory and computation costs of large pre-trained models. A key challenge in per-channel post-training quantization (PTQ) is selecting appropriate scaling factors to replace weight values with values from a scaled integer grid. Existing methods typically fix the scale at the outset via heuristic tuning or grid search. We propose Beacon, a simple and effective algorithm that eliminates the need for such manual tuning. Beacon performs per-channel PTQ directly using an unscaled grid and automatically determines the optimal scaling factors by exploiting the geometry of scalar quantization. It does not rely on back-propagation or large calibration sets. Despite its simplicity and tuning-free nature, Beacon achieves competitive performance compared to state-of-the-art methods, making it a practical solution for efficient model deployment.

Tables

Table 1. Table 1 . Weight-only quantization of DeiT-B.

	Beacon w/o E.C. (before/after LN)
1.58-bit(K=6)	67.49 / 72.82
2-bit(K=6)	75.01 / 77.19
3-bit(K=5)	80.53 / 80.69
4-bit(K=4)	81.35 / 81.40
runtime	$1-2\times$

Table 2. Table 2 . Accuracy Drop(%) Comparison on DeiT-B.

	2-bit	3-bit	4-bit
GPTQ (be/af LN)	20.48 / 15.15	1.81 / 1.56	0.42 / 0.32
COMQ	4.85	1.52	0.59
Beacon	4.55	1.05	0.34

Table 3. Table 3 . Accuracy(%) on DeiT-III-L.

	w/o E.C.	w/ E.C.	GPTQ	GPTQ*
2-bit	80.67	80.76	78.10	78.28
1.58-bit	77.07	77.32	46.25	72.74

Equations

$$\mathcal{Q}(\star )=c\cdot \left(\text{clip}\left(\lceil \frac{\star }{c}-z\rfloor ;0,2^b-1\right)+z\right),$$

$$z=\lceil \frac{\min (w)}{\max (w)-\min (w)}\cdot (2^b-1)\rfloor$$

$$\underset{Q\in {\mathcal{A}}^{N\times N^{\prime }},s\in {\mathbb{R}}^{N^{\prime }}}{\min }\Vert XW-XQ\mathrm{Diag}(s){\Vert }_F^2.$$

$$\underset{q\in {\mathcal{A}}^N,c\in \mathbb{R}}{\min }\Vert Xw-cXq{\Vert }_2^2.$$

$$c=\frac{⟨Xw,Xq⟩}{\Vert Xq{\Vert }^2}.$$

$$c^{\ast }=\frac{⟨Xw,X{q}^{\ast }⟩}{\Vert X{q}^{\ast }{\Vert }^2}.$$

$$\underset{q\in {\mathcal{A}}^N}{\min }{\Vert Xw-\frac{Xq(Xq{)}^{\top }Xw}{\Vert Xq{\Vert }^2}\Vert }^2.$$

$$\underset{q\in {\mathcal{A}}^N}{\max }\Vert {\mathcal{P}}_{Xq}(Xw){\Vert }^2=\underset{q\in {\mathcal{A}}^N}{\max }\frac{|⟨Xw,Xq⟩{|}^2}{\Vert Xq{\Vert }^2}.$$

$$\underset{q\in {\mathcal{A}}^N}{\max }|\cos \angle (Xw,Xq)|,$$

$$\underset{q\in {\mathcal{A}}^N}{\max }\cos \angle (Xw,Xq).$$

$$q_{t}^{(0)}\in\arg\max_{p\in\mathcal{A}}\ \cos\angle\!\big{(}X_{\leq t}w_{\leq t},\,X_{\leq t-1}q_{\leq t-1}^{(0)}+X_{t}p\big{)}.$$

$$e_0=\cos \angle (Xw,X{q}^{(0)}).$$

$$\arg\max_{p\in\mathcal{A}}\ \cos\angle\!\big{(}Xw,\,X_{\leq t-1}q_{\leq t-1}^{(\ell)}+X_{t}p+X_{\geq t+1}q_{\geq t+1}^{(\ell-1)}\big{)}.$$

$$e_{\ell }=\cos \angle (Xw,X{q}^{(\ell )}).$$

$$\underset{Q\in {\mathcal{A}}^{N\times N^{\prime }},s\in {\mathbb{R}}^{N^{\prime }}}{\min }\Vert XW-\tilde{X}Q\mathrm{Diag}(s){\Vert }_F^2.$$

$$q$$

$$=\underset{p\in {\mathcal{A}}^N}{\mathrm{argmax}}\frac{⟨(U^{\top }X)w,Rp⟩}{\Vert U^{\top }Xw\Vert \cdot \Vert Rp\Vert }.$$

Full text

This work was partially supported by NSF grant DMS-2410717.

Shihao Zhang is with the Department of Mathematics, UC San Diego (shz051@ucsd.edu). Rayan Saab is with the Department of Mathematics and HDSI, UC San Diego (rsaab@ucsd.edu).

1. Introduction

Recent deep neural networks—most notably large language models (LLMs)—have massive computational and memory demands. Quantization [7, 11, 17] has emerged as a mainstream compression technique [3, 5, 25] for deploying LLMs on resource-constrained devices and in other resource-constrained settings. By reducing the number of bits, or bit width, used to represent weights or activations, quantization lowers storage requirements, memory bandwidth, and computation costs. Post-training quantization (PTQ) [6, 24, 2, 23] is a particularly attractive approach for its simplicity. It avoids backpropagation and adapts pre-trained models usually in a single pass or in very few passes over a small calibration data set. So, it incurs far less computational overhead than quantization-aware training (QAT) [10, 19, 27], where quantized models are trained directly with gradient-based methods.

In practice, PTQ is most commonly implemented with scalar quantization, where each weight is ultimately represented by a value from a finite scalar alphabet, such as a uniform grid. While modern scalar PTQ algorithms such as GPTQ [6], GPFQ [14, 24], and Qronos [26] incorporate dependencies across weights during the quantization process, the quantization function itself still acts coordinatewise, mapping real values to scalar representatives. Scalar methods have become the industry standard and perform reliably at bit widths of 4 or more. However, pushing them into the ultra-low bit regime ($<3$ bits) remains difficult, even with transformation-based enhancements [20, 12, 1, 13]. An alternative is vector quantization (VQ), where groups of weights are jointly mapped to codewords from a vector codebook rather than to a scalar alphabet. VQ offers accuracy gains at ultra-low precision but at the expense of higher complexity and reduced deployment efficiency. In contrast, our method remains within the scalar quantization paradigm, with its minimal inference overhead, while achieving strong accuracy improvements at ultra-low bit widths.

Notation. We denote the weight matrix of a layer by $W\in\mathbb{R}^{N\times N^{\prime}}$, where each of the $N^{\prime}$ columns is an $N$-dimensional channel $w\in\mathbb{R}^{N}$. Given a vector $v\in\mathbb{R}^{n}$, we use $v_{i}$ for its $i$-th entry, $v_{\geq j}$ for the subvector $(v_{j},\dots,v_{n})^{\top}$, and we define $v_{\leq j}$ analogously. $\|v\|$ is the Euclidean norm of $v$. Given a matrix $A\in\mathbb{R}^{m\times n}$, we use $A_{i}$ to denote its $i$-th column. We use $A_{\leq j}$ to denote the submatrix $(A_{1},\dots,A_{j})$.

The standard $b$-bit integer grid is $\{0,1,\dots,2^{b}\!-\!1\}$, and weight quantization uses its scaled and shifted version $\mathcal{A}_{c}=\{c\cdot(z+k):k=0,\dots,2^{b}\!-\!1\},$ where $c$ is the scaling factor and $z$ is the offset (zero point). In asymmetric quantization, $c$ is typically defined per channel on a scaled min–max grid, $c=\frac{\alpha\cdot\max(w)-\beta\cdot\min(w)}{2^{b}-1}.$ The associated round-to-nearest (RTN) operator $\mathcal{Q}$ is

[TABLE]

where $\text{clip}(x;a_{\min},a_{\max})=\min\{\max\{x,a_{\min}\},a_{\max}\}$.

This paper considers asymmetric per-channel PTQ. For each channel (column) $w\in\mathbb{R}^{N}$ of $W$, we fix its unscaled integer grid $\mathcal{A}=\{z+k:k=0,\dots,2^{b}\!-\!1\},$ where we follow the standard choice of zero point

[TABLE]

throughout this paper. Each channel is quantized using a scaling factor $c\in\mathbb{R}$. Collecting these gives a vector $s\in\mathbb{R}^{N^{\prime}}$ of per-channel scaling factors for $W$.

Related Work. In per-channel PTQ, accurate scaling is critical for preserving model quality, particularly at ultra-low bit widths. Most scalar PTQ rounding methods [24, 6, 2] determine $s$ by tuning $\alpha$ and $\beta$ once at initialization and keeping them fixed thereafter. A recent exception is the method of Zhang et al. [22], which updates $s$ during its iterations but is highly sensitive to the initial choices of $\alpha$ and $\beta$. Other approaches attempt to refine scaling through trial-and-error, via grid search over a finite range [26, 21], or through simple heuristics, such as line search over the mean squared error between full-precision and quantized weights [1] or pre-activations [8].

Contributions. To our knowledge, no backpropagation-free algorithm currently performs per-channel quantization while automatically determining scaling factors. They all require a separate selection step for the scaling factors. In this work, we propose Beacon, an algorithm that carries out per-channel PTQ directly on the unscaled $b$-bit grid $\mathcal{A}=\{z+k:k=0,\dots,2^{b}\!-\!1\}$ and determines the optimal scaling at the end by exploiting the geometry of scalar quantization. Like other state-of-the-art PTQ methods, Beacon requires only a small calibration set and avoids backpropagation.

2. Preliminaries on Asymmetric Per-channel Quantization

Given a calibration matrix $X\in\mathbb{R}^{m\times N}$, a PTQ algorithm typically seeks a quantized weight matrix $Q$ that minimizes the layer-wise reconstruction error. Specifically, in per-channel quantization, each column $w\in\mathbb{R}^{N}$ of $W$ is associated with its own scaling factor, so that each column of $Q$ is drawn from an integer grid $\mathcal{A}$ and rescaled by a diagonal matrix of scales $s\in\mathbb{R}^{N^{\prime}}$. Thus the goal is to solve

[TABLE]

Here we slightly abuse our notation by using the same symbol $\mathcal{A}$ for all columns, when each column has its own zero point given by (1). Because the Frobenius norm decomposes as a sum of squared column errors, the problem separates across channels, which can be handled in parallel. Thus, it suffices to study a single column $w\in\mathbb{R}^{N}$, leading to the per-channel objective

[TABLE]

Although this problem is NP-hard in general [9], the subproblem of optimizing $c$ for a fixed $q$ admits a closed-form solution.

Proposition 2.1**.**

For any $q$, the optimal $c$ associated with the least squres objective in (3) is

[TABLE]

Proof.

Equation 3 is a least square problem in the one-dimensional real variable $c$ when $q$ is fixed. Taking derivatives with respect to $c$, the optimality condition is $2\|Xq\|^{2}c-2\langle Xw,Xq\rangle=0$, which implies the result. ∎

Corollary 2.2**.**

The global optimizer $(c^{*},q^{*})$ to (3) must satisfy the fixed point equation

[TABLE]

3. Beacon: PTQ with Automatic Per-channel Scaling

By Proposition 2.1, the optimal scaling constant $c$ for a chosen $q$ is uniquely given by (4). Substituting this into (3) eliminates $c$ and yields the equivalent objective

[TABLE]

Since $Xw-\tfrac{Xq(Xq)^{\top}Xw}{\|Xq\|^{2}}=(I-\mathcal{P}_{Xq})(Xw),$ where $\mathcal{P}_{Xq}=\frac{1}{\|Xq\|^{2}}Xq(Xq)^{\top}$ is the projection onto $\mathrm{span}(Xq)$, the problem reduces to

[TABLE]

Since $Xw$ is fixed, the objective is further equivalent to

[TABLE]

where $\cos\angle(Xw,Xq)=\frac{\langle Xw,Xq\rangle}{\|Xw\|\cdot\|Xq\|}.$ We can drop the absolute value and attempt to solve

[TABLE]

This formulation reveals the geometry of scalar quantization: aligning the directions of $Xq$ and $Xw$ is all you need.

**Beacon. **Inspired by the greedy path-following algorithm of Lybrand and Saab [14], Beacon starts by adopting a greedy approach to sequentially assign each $q_{i}\in\mathcal{A}$ for $i\in[N]$. Suppose $q_{1}^{(0)},\dots,q_{t-1}^{(0)}$ have already been chosen. The next coordinate is initialized by

[TABLE]

This produces an initial vector $q^{(0)}\in\mathcal{A}^{N}$ with objective value

[TABLE]

We then refine $q$ by cyclically updating each coordinate. At step $t$ of the $\ell$th cycle (loop), with all other coordinates fixed, the update rule is that $q_{t}^{(\ell)}$ must belong to

[TABLE]

After the $\ell$-th full sweep of updates, we denote the resulting vector by $q^{(\ell)}\in\mathcal{A}^{N}$ and the corresponding objective value by

[TABLE]

Proposition 3.1**.**

The sequence $(e_{\ell})_{\ell=0}^{\infty}$ converges in a finite number of iterations.

Proof.

By design of the $q$ update procedure, $(e_{\ell})_{\ell=0}^{\infty}$ is a non-decreasing sequence satisfying $0\leq e_{\ell}\leq 1$. By the monotone convergence theorem, it must converge. Moreover, since $q\in\mathcal{A}^{N}$, the sequence $(e_{\ell})_{\ell=0}^{\infty}$ can take at most $|\mathcal{A}|^{N}$ distinct values. Let $\mathcal{I}=\{\ell\in\mathbb{N}:e_{\ell}>e_{\ell-1}\}$, then $\mathcal{I}$ must be a finite subset of natural numbers. Let $\ell^{*}$ be the maximal index for which $e_{\ell}>e_{\ell-1}$, and we have $e_{L}=e_{\ell^{*}}$ for all $L\geq\ell^{*}+1$. ∎

Our experiments suggest that the quality of the quantized model improves during the first few $\ell$-loops and then plateaus, with the best results typically reached after 4–6 loops. The final step of Beacon is to use the resulting $q$ to compute the optimal scaling factor $c\in\mathbb{R}$ via (4).

We also empirically observe that sorting the columns of $X$ by increasing $\ell_{2}$ norm order in the initial assignment of $q^{(0)}$ (and by decreasing $\ell_{2}$ norm order in the refinement updates) improve the result upon natural index order. We present an intuition behind this observation. Let $u=\frac{X_{\leq t}w_{\leq t}}{\|X_{\leq t}w_{\leq t}\|}$ and $v=X_{\leq t-1}q_{\leq t-1}^{(0)}$. When determining $q_{t}^{(0)}$, we are selecting $p\in\mathcal{A}$ to maximize $\langle u,\frac{v+X_{t}p}{\|v+X_{t}p\|}\rangle$. If $\|X_{t}\|$ is dominated by $\|v\|$, $\frac{v+X_{t}p}{\|v+X_{t}p\|}$ will be largely determined by $\frac{v}{\|v\|}$, leaving little room for optimization. Thus, having the columns of $X$ in increasing $\ell_{2}$ norm order aligns better with Beacon.

**Memory Efficient Implementation. **We observe that the angle between $Xw$ and $Xq$ is rotation invariant. Let $X=UR$ be the QR decomposition of $X$. Then we have $\cos\angle(Xw,Xq)=\dfrac{\langle Rw,Rq\rangle}{\|Rw\|\cdot\|Rq\|}$, which reduces the problem from dealing with an often very tall matrix $X\in\mathbb{R}^{m\times N}$ to a square matrix $R\in\mathbb{R}^{N\times N}$.

**Handling Error Accumulation. ** Quantizing weights in earlier layers affects the inputs to subsequent layers. Let $X\in\mathbb{R}^{m\times N}$ denote the calibration set of $m$ samples (e.g., tokens) from the original pre-trained model, and let $\widetilde{X}\in\mathbb{R}^{m\times N}$ denote its counterpart from the partially quantized model. To account for the propagation of quantization error [26], one can address the mismatch between $X$ and $\widetilde{X}$ by approximately solving

[TABLE]

Beacon can be generalized in a memory-efficient way to handle distinct inputs $X$ and $\widetilde{X}$. Let $\widetilde{X}=UR$ be the QR decomposition of $\widetilde{X}$. Then we seek

[TABLE]

Thus, the problem again reduces from approximately solving $\max_{q\in\mathcal{A}^{N}}\ \cos\angle(Xw,\widetilde{X}q)$ to $\max_{q\in\mathcal{A}^{N}}\ \cos\angle(U^{\top}Xw,Rq),$ so that we work with the square matrices $U^{\top}X$ and $R$ rather than the potentially tall matrices $X$ and $\widetilde{X}$. We refer to this variant as Beacon with error correction. Its memory-efficient implementation is summarized in the algorithm below. In the special case without error correction, there is only one input $X=UR$, and we simply set $L=\widetilde{L}=R$.

**Normalization Tuning. **A common practice in PTQ is to add a lightweight training step to tune the unquantized parameters in batch normalization (BN) or layer normalization (LN) layers, helping to compensate for quantization error. We evaluate the effect of normalization tuning in our experiments.

4. Experiments

We first test Beacon on the DeiT-B vision transformer model ([15], 86 million parameters) on the benchmark ImageNet classification task. To that end, we use DeiT-B ($patch16\_224$ version) from the Hugging Face timm library [18]. We focus on ILSVRC-2012 [4], a 1000-category dataset with 1.28 million training images and 50 thousand validation images. All images in ILSVRC-2012 are preprocessed in the standard manner by resizing each image to $256\times 256$ and using the normalized $224\times 224$ center crop. We evaluate top-1 accuracy of the quantized models on the entire validation set. The original accuracy of DeiT-B is $81.74\%$. We use batch size $2048$ to generate the calibration data. We ran our experiments on a single Nvidia A100 GPU with 80G GPU memory.

Table 1 displays the result of quantizing DeiT-B via Beacon without error correction, as the number of bits varies. $1.58$-bit quantization means the grid is a scaling and shifting of $\{0,1,2\}$ per channel. We display results before and after a lightweight LN tuning after the whole model has been quantized. $K=\ell_{\max}$ is the number of loops we applied, and is fixed for each row. We remark that the best $K$, numerically, for each variant (whether with or without error correction or LN tuning, the bit width) slightly differ, but are typically $K=4,5,$ or $6$. The runtime, compared to GPTQ with the same set up and machine, is reported in the last row. The LN tuning step only adds a small extra cost when training for $1$ epoch with $100\!-\!200$ data batches with batch size $128$.

We compare Beacon to a recent state-of-the-art method for quantizing vision models, namely COMQ [22], and the standard baseline method GPTQ [6] in Table 2, again evaluated on DeiT-B. We implement GPTQ with asymmetric quantization on a standard per-channel min-max grid ($\alpha\!=\!\beta\!=\!1$, $z$ given by (1)) and use the reported result in Zhang et al. [22]. As the original accuracy of DeiT-B reported in Zhang et al. [22] is $81.99\%$, which differs slightly from the $81.74\%$ we observe, here we compare the accuracy drop associated with each algorithm. The results suggest that fine grid tuning is critical to $<3$ bit quantization, further highlighting the benefit of Beacon as a tuning free method for setting the scale. Indeed, Beacon achieves the best performance for the challenging 2-bit case. We also note from the open-sourced code that Zhang et al. [22] used the entire training dataset for LN tuning and carefully initialize the scaling parameters to be $\alpha\!=\!\beta\!=\!0.65$ for 2-bit case. In contrast, Beacon only uses a few batches for LN tuning and, by construction, requires no hyper-parameter tuning.

Table 3 displays the result of quantizing DeiT-III-L ([16], 304 million parameters, original accuracy 84.59%) with LN tuning for all methods. The first two columns are for Beacon with $K=6$, with and without error correction respectively. GPTQ is still implemented with asymmetric quantization on a per-channel min-max grid.

GPTQ* uses Beacon to generate its scaling factors: Given a channel $w$ and its unscaled grid $\mathcal{A}=\{z\!+\!k:k=0,\dots,2^{b}\!-\!1\}$ with $z=\left\lceil\frac{\min(w)}{\max(w)-\min(w)}\cdot(2^{b}-1)\right\rfloor$, we first run Beacon without error correction, giving us a scaling factor $\hat{c}$ for this channel to replace the choice $c=\frac{\max(w)-\min(w)}{2^{b}-1}$ used in GPTQ. We observe that the scaling generated by Beacon significantly improves the 1.58-bit GPTQ quantized model to a usable quality, although Beacon still outperforms it. Table 1 and Table 3 both demonstrate that Beacon yields a usable model even in the extremely challenging 1.58-bit quantization setting.

5. Conclusion

We introduced Beacon, a simple and tuning-free algorithm for post-training quantization. Unlike existing approaches that require heuristic scale selection or iterative search, Beacon directly quantizes weights using an unscaled grid and infers optimal scaling factors per-channel after quantization. This streamlines the PTQ process and avoids dependence on hyperparameter tuning or back-propagation. Beacon yields models that retain compatibility with standard hardware and matches the performance of more complex state-of-the-art methods, despite its minimal calibration and computational overhead. We believe that this makes it a simple and effective solution for compressing large models in resource-constrained environments.