Win the Lottery Ticket via Fourier Analysis: Frequencies Guided Network Pruning
Yuzhang Shang, Bin Duan, Ziliang Zong, Liqiang Nie, Yan Yan

TL;DR
This paper introduces a Fourier analysis-based approach to guide network pruning, improving the effectiveness of magnitude-based pruning by analyzing frequency domain properties, and demonstrates its superiority through extensive experiments.
Contribution
It proposes a novel two-stage pruning method guided by Fourier analysis, enhancing network compression and performance recovery.
Findings
Fourier analysis explains MBP's generalization ability.
The proposed method outperforms traditional MBP algorithms.
Experiments on CIFAR datasets validate the approach.
Abstract
With the remarkable success of deep learning recently, efficient network compression algorithms are urgently demanded for releasing the potential computational power of edge devices, such as smartphones or tablets. However, optimal network pruning is a non-trivial task which mathematically is an NP-hard problem. Previous researchers explain training a pruned network as buying a lottery ticket. In this paper, we investigate the Magnitude-Based Pruning (MBP) scheme and analyze it from a novel perspective through Fourier analysis on the deep learning model to guide model designation. Besides explaining the generalization ability of MBP using Fourier transform, we also propose a novel two-stage pruning approach, where one stage is to obtain the topological structure of the pruned network and the other stage is to retrain the pruned network to recover the capacity using knowledge…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsVideo Analysis and Summarization · Advanced Image and Video Retrieval Techniques · Music and Audio Processing
MethodsPruning · Knowledge Distillation
