A One-Shot Reparameterization Method for Reducing the Loss of Tile Pruning on DNNs

TL;DR

This paper introduces TileTrans, a one-shot reparameterization technique that rearranges weights in DNNs to significantly reduce accuracy loss during tile pruning without retraining.

Contribution

The novel TileTrans method repermutes weight matrix rows or columns to preserve important elements, improving tile pruning accuracy without additional training.

Findings

Improves AlexNet accuracy by up to 17%.

Enhances ResNet-34 accuracy by 5%.

Seamlessly integrates with existing tile pruning methods.

Abstract

Recently, tile pruning has been widely studied to accelerate the inference of deep neural networks (DNNs). However, we found that the loss due to tile pruning, which can eliminate important elements together with unimportant elements, is large on trained DNNs. In this study, we propose a one-shot reparameterization method, called TileTrans, to reduce the loss of tile pruning. Specifically, we repermute the rows or columns of the weight matrix such that the model architecture can be kept unchanged after reparameterization. This repermutation realizes the reparameterization of the DNN model without any retraining. The proposed reparameterization method combines important elements into the same tile; thus, preserving the important elements after the tile pruning. Furthermore, TileTrans can be seamlessly integrated into existing tile pruning methods because it is a pre-processing method executed before pruning, which is orthogonal to most existing methods. The experimental results demonstrate that our method is essential in reducing the loss of tile pruning on DNNs. Specifically, the accuracy is improved by up to 17% for AlexNet while 5% for ResNet-34, where both models are pre-trained on ImageNet.