Uncertainty Makes It Stable: Curiosity-Driven Quantized Mixture-of-Experts

TL;DR

This paper introduces a curiosity-driven quantized Mixture-of-Experts framework that maintains high accuracy and stability in resource-constrained neural network deployment by leveraging Bayesian uncertainty for dynamic expert routing.

Contribution

It presents a novel uncertainty-based routing mechanism in quantized Mixture-of-Experts models, improving accuracy, stability, and interpretability for edge AI applications.

Findings

4-bit quantization achieves 99.9% of full-precision F1 score with 4x compression.

Curiosity-driven routing reduces variance by up to 85% across datasets.

High-precision experts automatically handle uncertain samples, enhancing stability.

Abstract

Deploying deep neural networks on resource-constrained devices faces two critical challenges: maintaining accuracy under aggressive quantization while ensuring predictable inference latency. We present a curiosity-driven quantized Mixture-of-Experts framework that addresses both through Bayesian epistemic uncertainty-based routing across heterogeneous experts (BitNet ternary, 1-16 bit BitLinear, post-training quantization). Evaluated on audio classification benchmarks (ESC-50, Quinn, UrbanSound8K), our 4-bit quantization maintains 99.9 percent of full-precision F1 (0.858 vs 0.859) with 4x compression and 31 percent energy savings versus 8-bit, while both achieve statistical parity with full precision (p > 0.05). Crucially, curiosity-driven routing simultaneously improves accuracy and stability: on Quinn, F1 increases from 0.802 to 0.809 while cross-fold variance drops by 85 percent (p < 0.001, Levene's test), with reductions of 50 to 94 percent across datasets. The routing is self-organizing, with the high-precision 8-bit expert automatically receiving the most uncertain samples (20 percent lower confidence, p < 0.001), while lightweight experts handle easier inputs. Datasets with already low baseline variance show no artificial stability gain, confirming the mechanism targets genuine epistemic uncertainty rather than overfitting routing decisions. At 1.2M parameters, the framework provides interpretable, precision-aware routing suitable for safety-sensitive edge deployments where both accuracy and predictability are critical.