FIT: A Metric for Model Sensitivity

TL;DR

This paper introduces FIT, a fast, information geometric metric that predicts a neural network's sensitivity to quantization, enabling efficient layer-wise mixed-precision optimization without retraining.

Contribution

FIT combines Fisher information with quantization modeling to estimate network performance loss due to quantization, providing a novel, fast, and unified sensitivity metric.

Findings

FIT accurately predicts network performance across many quantization setups.

It outperforms existing methods in speed and convergence.

Effective for layer-wise mixed-precision quantization.

Abstract

Model compression is vital to the deployment of deep learning on edge devices. Low precision representations, achieved via quantization of weights and activations, can reduce inference time and memory requirements. However, quantifying and predicting the response of a model to the changes associated with this procedure remains challenging. This response is non-linear and heterogeneous throughout the network. Understanding which groups of parameters and activations are more sensitive to quantization than others is a critical stage in maximizing efficiency. For this purpose, we propose FIT. Motivated by an information geometric perspective, FIT combines the Fisher information with a model of quantization. We find that FIT can estimate the final performance of a network without retraining. FIT effectively fuses contributions from both parameter and activation quantization into a single metric. Additionally, FIT is fast to compute when compared to existing methods, demonstrating favourable convergence properties. These properties are validated experimentally across hundreds of quantization configurations, with a focus on layer-wise mixed-precision quantization.