Quantization Robustness of Monotone Operator Equilibrium Networks

TL;DR

This paper analyzes how weight quantization affects the convergence and accuracy of monotone operator equilibrium networks, providing theoretical bounds and experimental validation for different quantization levels.

Contribution

It introduces a spectral perturbation analysis of quantization effects and establishes convergence guarantees and error bounds for these networks under low-precision weights.

Findings

Convergence is guaranteed if spectral-norm perturbation is below the monotonicity margin.

Quantization at three or four bits causes divergence, while five bits or more maintain convergence.

Quantization-aware training can recover convergence at four bits.

Abstract

Monotone operator equilibrium networks are implicit-layer models whose output is the unique equilibrium of a monotone operator, guaranteeing existence, uniqueness, and convergence. When deployed on low-precision hardware, weights are quantized, potentially destroying these guarantees. We analyze weight quantization as a spectral perturbation of the underlying monotone inclusion. Convergence of the quantized solver is guaranteed whenever the spectral-norm weight perturbation is smaller than the monotonicity margin; the displacement between quantized and full-precision equilibria is bounded in terms of the perturbation size and margin; and a condition number characterizing the ratio of the operator norm to the margin links quantization precision to forward error. MNIST experiments confirm a phase transition at the predicted threshold: three- and four-bit post-training quantization diverge, while five-bit and above converge. The backward-pass guarantee enables quantization-aware training, which recovers provable convergence at four bits.