SINF: Semantic Neural Network Inference with Semantic Subgraphs

TL;DR

SINF introduces a method to create semantic subgraphs in DNNs using a new Discriminative Capability Score, significantly reducing inference time and energy consumption with minimal accuracy loss across multiple models and datasets.

Contribution

The paper presents SINF, a novel semantic inference approach that leverages a new DCS to prune DNNs efficiently, outperforming existing pruning methods in speed and energy efficiency.

Findings

Up to 35% reduction in inference time for VGG19.

Energy efficiency improved by 51% on Raspberry Pi.

Minimal accuracy loss of less than 4% on CIFAR100.

Abstract

This paper proposes Semantic Inference (SINF) that creates semantic subgraphs in a Deep Neural Network(DNN) based on a new Discriminative Capability Score (DCS) to drastically reduce the DNN computational load with limited performance loss.~We evaluate the performance SINF on VGG16, VGG19, and ResNet50 DNNs trained on CIFAR100 and a subset of the ImageNet dataset. Moreover, we compare its performance against 6 state-of-the-art pruning approaches. Our results show that (i) on average, SINF reduces the inference time of VGG16, VGG19, and ResNet50 respectively by up to 29%, 35%, and 15% with only 3.75%, 0.17%, and 6.75% accuracy loss for CIFAR100 while for ImageNet benchmark, the reduction in inference time is 18%, 22%, and 9% for accuracy drop of 3%, 2.5%, and 6%; (ii) DCS achieves respectively up to 3.65%, 4.25%, and 2.36% better accuracy with VGG16, VGG19, and ResNet50 with respect to existing discriminative scores for CIFAR100 and the same for ImageNet is 8.9%, 5.8%, and 5.2% respectively. Through experimental evaluation on Raspberry Pi and NVIDIA Jetson Nano, we show SINF is about 51% and 38% more energy efficient and takes about 25% and 17% less inference time than the base model for CIFAR100 and ImageNet.