AdaptGrad: Adaptive Sampling to Reduce Noise
Linjiang Zhou, Chao Ma, Zepeng Wang, Libing Wu, Xiaochuan Shi

TL;DR
AdaptGrad introduces an adaptive sampling method for gradient smoothing that effectively reduces noise in gradient explanations, improving interpretability without manual hyper-parameter tuning.
Contribution
The paper reinterprets SmoothGrad as a convolution process and proposes AdaptGrad, an adaptive method that automatically adjusts noise levels for better gradient noise reduction.
Findings
AdaptGrad significantly reduces noise compared to baseline methods.
It improves the clarity and reliability of gradient-based explanations.
The method is simple, universal, and enhances interpretability.
Abstract
Gradient Smoothing is an efficient approach to reducing noise in gradient-based model explanation method. SmoothGrad adds Gaussian noise to mitigate much of these noise. However, the crucial hyper-parameter in this method, the variance of Gaussian noise, is set manually or with heuristic approach. However, it results in the smoothed gradients still containing a certain amount of noise. In this paper, we aim to interpret SmoothGrad as a corollary of convolution, thereby re-understanding the gradient noise and the role of from the perspective of confidence level. Furthermore, we propose an adaptive gradient smoothing method, AdaptGrad, based on these insights. Through comprehensive experiments, both qualitative and quantitative results demonstrate that AdaptGrad could effectively reduce almost all the noise in vanilla gradients compared with baselines methods. AdaptGrad…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsModel Reduction and Neural Networks · Advanced Control Systems Optimization · Reservoir Engineering and Simulation Methods
MethodsSparse Evolutionary Training
