Dynamic Influence Tracker: Measuring Time-Varying Sample Influence During Training
Jie Xu, Zihan Wu

TL;DR
The paper introduces Dynamic Influence Tracker (DIT), a method that measures how individual training sample influence varies over time during training, revealing insights into learning phases and improving corrupted sample detection.
Contribution
DIT is the first approach to capture time-varying sample influence during training, providing theoretical guarantees and outperforming existing static influence measurement methods.
Findings
Samples exhibit different influence patterns over training stages.
Weak correlation between early and late influence indicates distinct learning phases.
Analyzing influence during convergence improves corrupted sample detection accuracy.
Abstract
Existing methods for measuring training sample influence on models only provide static, overall measurements, overlooking how sample influence changes during training. We propose Dynamic Influence Tracker (DIT), which captures the time-varying sample influence across arbitrary time windows during training. DIT offers three key insights: 1) Samples show different time-varying influence patterns, with some samples important in the early training stage while others become important later. 2) Sample influences show a weak correlation between early and late stages, demonstrating that the model undergoes distinct learning phases with shifting priorities. 3) Analyzing influence during the convergence period provides more efficient and accurate detection of corrupted samples than full-training analysis. Supported by theoretical guarantees without assuming loss convexity or model convergence,…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsSports Performance and Training · Gaussian Processes and Bayesian Inference · Time Series Analysis and Forecasting
