An Once-for-All Budgeted Pruning Framework for ConvNets Considering Input Resolution

TL;DR

This paper introduces OFARPruning, a novel framework for efficiently pruning ConvNets considering input resolution, enabling the creation of adaptable, high-accuracy networks with minimal training for various edge device constraints.

Contribution

The proposed OFARPruning framework efficiently finds multiple compact network structures considering input resolution, outperforming existing methods in accuracy and efficiency with only one training process.

Findings

OFARPruning achieves 1-2% higher accuracy than US-Net and MutualNet.

It matches or exceeds traditional pruning methods' accuracy with less FLOPs.

The method effectively adapts networks to different input resolutions and FLOPs constraints.

Abstract

We propose an efficient once-for-all budgeted pruning framework (OFARPruning) to find many compact network structures close to winner tickets in the early training stage considering the effect of input resolution during the pruning process. In structure searching stage, we utilize cosine similarity to measure the similarity of the pruning mask to get high-quality network structures with low energy and time consumption. After structure searching stage, our proposed method randomly sample the compact structures with different pruning rates and input resolution to achieve joint optimization. Ultimately, we can obtain a cohort of compact networks adaptive to various resolution to meet dynamic FLOPs constraints on different edge devices with only once training. The experiments based on image classification and object detection show that OFARPruning has a higher accuracy than the once-for-all compression methods such as US-Net and MutualNet (1-2% better with less FLOPs), and achieve the same even higher accuracy as the conventional pruning methods (72.6% vs. 70.5% on MobileNetv2 under 170 MFLOPs) with much higher efficiency.