Reconstruction Task Finds Universal Winning Tickets
Ruichen Li, Binghui Li, Qi Qian, Liwei Wang

TL;DR
This paper introduces a novel pruning method that incorporates image reconstruction to produce more transferable and effective pruned neural networks for diverse downstream tasks in computer vision.
Contribution
The paper proposes integrating pixel-level image reconstruction into pruning to enhance transferability of pruned models across various downstream tasks.
Findings
Outperforms state-of-the-art pruning methods on benchmark tasks.
Improves transferability of pruned models to complex downstream tasks.
Demonstrates effectiveness of reconstruction-based pruning in computer vision.
Abstract
Pruning well-trained neural networks is effective to achieve a promising accuracy-efficiency trade-off in computer vision regimes. However, most of existing pruning algorithms only focus on the classification task defined on the source domain. Different from the strong transferability of the original model, a pruned network is hard to transfer to complicated downstream tasks such as object detection arXiv:arch-ive/2012.04643. In this paper, we show that the image-level pretrain task is not capable of pruning models for diverse downstream tasks. To mitigate this problem, we introduce image reconstruction, a pixel-level task, into the traditional pruning framework. Concretely, an autoencoder is trained based on the original model, and then the pruning process is optimized with both autoencoder and classification losses. The empirical study on benchmark downstream tasks shows that the…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsAdvanced Neural Network Applications · Image Processing Techniques and Applications · Domain Adaptation and Few-Shot Learning
MethodsPruning
