Improved Techniques for Quantizing Deep Networks with Adaptive Bit-Widths

TL;DR

This paper introduces two novel techniques for training adaptive bit-width quantized deep networks, enabling flexible inference across various resource constraints with improved accuracy and efficiency.

Contribution

It proposes a collaborative knowledge transfer strategy and a dynamic block swapping method for better training of adaptive quantized networks.

Findings

Outperforms state-of-the-art adaptive quantization methods.

Effective on multiple image and video classification benchmarks.

Enables flexible inference with a single network.

Abstract

Quantizing deep networks with adaptive bit-widths is a promising technique for efficient inference across many devices and resource constraints. In contrast to static methods that repeat the quantization process and train different models for different constraints, adaptive quantization enables us to flexibly adjust the bit-widths of a single deep network during inference for instant adaptation in different scenarios. While existing research shows encouraging results on common image classification benchmarks, this paper investigates how to train such adaptive networks more effectively. Specifically, we present two novel techniques for quantizing deep neural networks with adaptive bit-widths of weights and activations. First, we propose a collaborative strategy to choose a high-precision teacher for transferring knowledge to the low-precision student while jointly optimizing the model with all bit-widths. Second, to effectively transfer knowledge, we develop a dynamic block swapping method by randomly replacing the blocks in the lower-precision student network with the corresponding blocks in the higher-precision teacher network. Extensive experiments on multiple image classification datasets including video classification benchmarks for the first time, well demonstrate the efficacy of our approach over state-of-the-art methods.