Technical Report: NEMO DNN Quantization for Deployment Model

TL;DR

This report introduces a formal framework for layer-wise DNN quantization tailored for deployment, detailing the NEMO framework's representations, especially the IntegerDeployable model enabling integer-only inference without real numbers.

Contribution

It provides a formal definition of DNN quantization representations within NEMO, notably the IntegerDeployable model for integer-only inference in deployment scenarios.

Findings

Defines four DNN representations used in NEMO.

Introduces IntegerDeployable enabling integer-only inference.

Formalizes layer-wise quantization for deployment.

Abstract

This technical report aims at defining a formal framework for Deep Neural Network (DNN) layer-wise quantization, focusing in particular on the problems related to the final deployment. It also acts as a documentation for the NEMO (NEural Minimization for pytOrch) framework. It describes the four DNN representations used in NEMO (FullPrecision, FakeQuantized, QuantizedDeployable and IntegerDeployable), focusing in particular on a formal definition of the latter two. An important feature of this model, and in particular the IntegerDeployable representation, is that it enables DNN inference using purely integers - without resorting to real-valued numbers in any part of the computation and without relying on an explicit fixed-point numerical representation.