Quantized and Interpretable Learning Scheme for Deep Neural Networks in Classification Task

TL;DR

This paper presents a combined saliency-guided training and quantization approach using PACT to create resource-efficient, interpretable deep neural networks that maintain high accuracy for image classification tasks.

Contribution

It introduces a novel training scheme that integrates saliency-guided learning with quantization-aware training to enhance interpretability and efficiency without sacrificing accuracy.

Findings

Models are more resource-efficient with maintained accuracy.

Saliency maps are clearer and more focused in quantized models.

The approach is validated on MNIST and CIFAR-10 datasets.

Abstract

Deep learning techniques have proven highly effective in image classification, but their deployment in resourceconstrained environments remains challenging due to high computational demands. Furthermore, their interpretability is of high importance which demands even more available resources. In this work, we introduce an approach that combines saliency-guided training with quantization techniques to create an interpretable and resource-efficient model without compromising accuracy. We utilize Parameterized Clipping Activation (PACT) to perform quantization-aware training, specifically targeting activations and weights to optimize precision while minimizing resource usage. Concurrently, saliency-guided training is employed to enhance interpretability by iteratively masking features with low gradient values, leading to more focused and meaningful saliency maps. This training procedure helps in mitigating noisy gradients and yields models that provide clearer, more interpretable insights into their decision-making processes. To evaluate the impact of our approach, we conduct experiments using famous Convolutional Neural Networks (CNN) architecture on the MNIST and CIFAR-10 benchmark datasets as two popular datasets. We compare the saliency maps generated by standard and quantized models to assess the influence of quantization on both interpretability and classification accuracy. Our results demonstrate that the combined use of saliency-guided training and PACT-based quantization not only maintains classification performance but also produces models that are significantly more efficient and interpretable, making them suitable for deployment in resource-limited settings.