Stabilizing Linear Passive-Aggressive Online Learning with Weighted Reservoir Sampling

TL;DR

This paper introduces a weighted reservoir sampling method to stabilize online learning algorithms like PA classifiers by maintaining a high-quality ensemble of solutions, improving accuracy especially in the presence of outliers.

Contribution

The paper proposes a novel weighted reservoir sampling technique that enhances the stability and accuracy of online learning models without additional data passes or memory overhead.

Findings

WRS improves stability of online classifiers against outliers.

Ensemble approach outperforms standard online methods in experiments.

Risk of ensemble classifier is bounded by the regret of the base online method.

Abstract

Online learning methods, like the seminal Passive-Aggressive (PA) classifier, are still highly effective for high-dimensional streaming data, out-of-core processing, and other throughput-sensitive applications. Many such algorithms rely on fast adaptation to individual errors as a key to their convergence. While such algorithms enjoy low theoretical regret, in real-world deployment they can be sensitive to individual outliers that cause the algorithm to over-correct. When such outliers occur at the end of the data stream, this can cause the final solution to have unexpectedly low accuracy. We design a weighted reservoir sampling (WRS) approach to obtain a stable ensemble model from the sequence of solutions without requiring additional passes over the data, hold-out sets, or a growing amount of memory. Our key insight is that good solutions tend to be error-free for more iterations than bad solutions, and thus, the number of passive rounds provides an estimate of a solution's relative quality. Our reservoir thus contains $K$ previous intermediate weight vectors with high survival times. We demonstrate our WRS approach on the Passive-Aggressive Classifier (PAC) and First-Order Sparse Online Learning (FSOL), where our method consistently and significantly outperforms the unmodified approach. We show that the risk of the ensemble classifier is bounded with respect to the regret of the underlying online learning method.