A Comprehensive Study of Structural Pruning for Vision Models

TL;DR

This paper introduces PruningBench, a comprehensive benchmark for structural pruning in vision models, evaluating 16 methods across various models and tasks to standardize progress assessment.

Contribution

It provides the first unified benchmark and evaluation framework for structural pruning, including a platform for reproducibility and future method comparison.

Findings

Systematic evaluation of 16 pruning methods

Unified framework for diverse models and tasks

Accessible platform for benchmarking and reproducibility

Abstract

Structural pruning has emerged as a promising approach for producing more efficient models. Nevertheless, the community suffers from a lack of standardized benchmarks and metrics, leaving the progress in this area not fully comprehended. To fill this gap, we present the first comprehensive benchmark, termed PruningBench, for structural pruning. PruningBench showcases the following three characteristics: 1) PruningBench employs a unified and consistent framework for evaluating the effectiveness of diverse structural pruning techniques; 2) PruningBench systematically evaluates 16 existing pruning methods, encompassing a wide array of models (e.g., CNNs and ViTs) and tasks (e.g., classification and detection); 3) PruningBench provides easily implementable interfaces to facilitate the implementation of future pruning methods, and enables the subsequent researchers to incorporate their work into our leaderboards. We provide an online pruning platform for customizing pruning tasks and reproducing all results in this paper. Leaderboard results can also be available.