prunAdag: an adaptive pruning-aware gradient method
Margherita Porcelli, Giovanni Seraghiti, Philippe L. Toint

TL;DR
This paper introduces prunAdag, an adaptive gradient method that classifies variables for targeted updates, enabling effective sparsification and demonstrating competitive convergence and performance in various applications.
Contribution
It extends previous relevance-based methods by incorporating a pruning-aware adaptive gradient approach with proven convergence guarantees.
Findings
Proves convergence with a global rate of O(\u001dlog(k)/\u001d extrac{1}{\u001d ext extrac{1}{",
Demonstrates competitive performance in multiple numerical experiments.
Enables a posteriori sparsification of model parameters.
Abstract
A pruning-aware adaptive gradient method is proposed which classifies the variables in two sets before updating them using different strategies. This technique extends the ``relevant/irrelevant" approach of Ding (2019) and Zimmer et al. (2022) and allows a posteriori sparsification of the solution of model parameter fitting problems. The new method is proved to be convergent with a global rate of decrease of the averaged gradient's norm of the form . Numerical experiments on several applications show that it is competitive.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsNeural Networks and Applications · Video Analysis and Summarization · Advanced Image and Video Retrieval Techniques
