Augmenting Hessians with Inter-Layer Dependencies for Mixed-Precision Post-Training Quantization

TL;DR

This paper introduces a mixed-precision post-training quantization method that enhances layer sensitivity estimation by incorporating inter-layer dependencies, leading to significant latency reductions while preserving model accuracy.

Contribution

It proposes augmenting Hessian-based sensitivity analysis with inter-layer dependency information to improve quantization decisions in neural networks.

Findings

Achieves up to 33.28% latency reduction on BERT

Maintains model accuracy within 99.99% of baseline

Improves accuracy-latency trade-offs across multiple models

Abstract

Efficiently serving neural network models with low latency is becoming more challenging due to increasing model complexity and parameter count. Model quantization offers a solution which simultaneously reduces memory footprint and compute requirements. However, aggressive quantization may lead to an unacceptable loss in model accuracy owing to differences in sensitivity to numerical imperfection across different layers in the model. To address this challenge, we propose a mixed-precision post training quantization (PTQ) approach that assigns different numerical precisions to tensors in a network based on their specific needs, for a reduced memory footprint and improved latency while preserving model accuracy. Previous works rely on layer-wise Hessian information to determine numerical precision, but as we demonstrate, Hessian estimation is typically insufficient in determining an effective ordering of layer sensitivities. We address this by augmenting the estimated Hessian with additional information to capture inter-layer dependencies. We demonstrate that this consistently improves PTQ performance along the accuracy-latency Pareto frontier across multiple models. Our method combines second-order information and inter-layer dependencies to guide a bisection search, finding quantization configurations within a user-configurable model accuracy degradation range. We evaluate the effectiveness of our method on the ResNet50, MobileNetV2, and BERT models. Our experiments demonstrate latency reductions compared to a 16-bit baseline of $25.48\%$, $21.69\%$, and $33.28\%$ respectively, while maintaining model accuracy to within $99.99\%$ of the baseline model.