Practical Network Acceleration with Tiny Sets

TL;DR

This paper introduces PRACTISE, a novel method for accelerating neural networks using tiny training sets by dropping blocks based on recoverability, achieving significant latency reduction and better accuracy than previous methods.

Contribution

It proposes a block-dropping approach with a new recoverability metric and an algorithm that outperforms existing methods in few-shot network acceleration.

Findings

PRACTISE achieves 7% better accuracy at 22% latency reduction on ImageNet-1k.

Dropping blocks is more effective than filter-level pruning in few-shot scenarios.

The method generalizes well to data-free and out-of-domain settings.

Abstract

Due to data privacy issues, accelerating networks with tiny training sets has become a critical need in practice. Previous methods mainly adopt filter-level pruning to accelerate networks with scarce training samples. In this paper, we reveal that dropping blocks is a fundamentally superior approach in this scenario. It enjoys a higher acceleration ratio and results in a better latency-accuracy performance under the few-shot setting. To choose which blocks to drop, we propose a new concept namely recoverability to measure the difficulty of recovering the compressed network. Our recoverability is efficient and effective for choosing which blocks to drop. Finally, we propose an algorithm named PRACTISE to accelerate networks using only tiny sets of training images. PRACTISE outperforms previous methods by a significant margin. For 22% latency reduction, PRACTISE surpasses previous methods by on average 7% on ImageNet-1k. It also enjoys high generalization ability, working well under data-free or out-of-domain data settings, too. Our code is at https://github.com/DoctorKey/Practise.