COPS: Controlled Pruning Before Training Starts
Paul Wimmer, Jens Mehnert, Alexandru Condurache

TL;DR
COPS introduces a novel framework for combining multiple pruning scores through a linear programming approach, enabling more effective one-shot pruning of neural networks before training, leading to improved performance.
Contribution
The paper proposes a new method for combining arbitrary pruning scores via a linear program, enhancing one-shot pruning strategies for neural networks.
Findings
COPS outperforms existing pruning methods across various architectures.
The LP-based approach reduces computational complexity compared to general solvers.
Experimental results show improved accuracy in image classification tasks.
Abstract
State-of-the-art deep neural network (DNN) pruning techniques, applied one-shot before training starts, evaluate sparse architectures with the help of a single criterion -- called pruning score. Pruning weights based on a solitary score works well for some architectures and pruning rates but may also fail for other ones. As a common baseline for pruning scores, we introduce the notion of a generalized synaptic score (GSS). In this work we do not concentrate on a single pruning criterion, but provide a framework for combining arbitrary GSSs to create more powerful pruning strategies. These COmbined Pruning Scores (COPS) are obtained by solving a constrained optimization problem. Optimizing for more than one score prevents the sparse network to overly specialize on an individual task, thus COntrols Pruning before training Starts. The combinatorial optimization problem given by COPS is…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsAdvanced Neural Network Applications · Neural Networks and Applications · Machine Learning and ELM
MethodsPruning
