Bit-Mixer: Mixed-precision networks with runtime bit-width selection
Adrian Bulat, Georgios Tzimiropoulos

TL;DR
Bit-Mixer introduces a novel meta-quantized network that enables dynamic, runtime adjustment of layer bit-widths in mixed-precision networks without sacrificing accuracy, enhancing deployment flexibility.
Contribution
It is the first method to train a network allowing layer-wise bit-width changes at runtime, using transitional Batch-Norms and a 3-stage optimization process.
Findings
Enables flexible bit-width adjustment during inference.
Maintains high accuracy despite runtime bit-width changes.
Demonstrates effective deployment on devices with variable characteristics.
Abstract
Mixed-precision networks allow for a variable bit-width quantization for every layer in the network. A major limitation of existing work is that the bit-width for each layer must be predefined during training time. This allows little flexibility if the characteristics of the device on which the network is deployed change during runtime. In this work, we propose Bit-Mixer, the very first method to train a meta-quantized network where during test time any layer can change its bid-width without affecting at all the overall network's ability for highly accurate inference. To this end, we make 2 key contributions: (a) Transitional Batch-Norms, and (b) a 3-stage optimization process which is shown capable of training such a network. We show that our method can result in mixed precision networks that exhibit the desirable flexibility properties for on-device deployment without compromising…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsVLSI and Analog Circuit Testing · Software Testing and Debugging Techniques · Network Packet Processing and Optimization
