OLR-WAA: Adaptive and Drift-Resilient Online Regression with Dynamic Weighted Averaging

TL;DR

OLR-WAA is a hyperparameter-free online regression model that adaptively detects and responds to concept drift, maintaining high performance in evolving data streams without manual tuning.

Contribution

It introduces a novel adaptive, drift-resilient online regression approach with dynamic weighting and real-time drift detection, eliminating the need for fixed hyperparameters.

Findings

Matches batch regression in static settings

Outperforms or rivals state-of-the-art online models

Bridges performance gap in concept drift scenarios

Abstract

Real-world datasets frequently exhibit evolving data distributions, reflecting temporal variations and underlying shifts. Overlooking this phenomenon, known as concept drift, can substantially degrade the predictive performance of the model. Furthermore, the presence of hyperparameters in online models exacerbates this issue, as these parameters are typically fixed and lack the flexibility to dynamically adjust to evolving data. This paper introduces "OLR-WAA: An Adaptive and Drift-Resilient Online Regression with Dynamic Weighted Average", a hyperparameter-free model designed to tackle the challenges of non-stationary data streams and enable effective, continuous adaptation. The objective is to strike a balance between model stability and adaptability. OLR-WAA incrementally updates its base model by integrating incoming data streams, utilizing an exponentially weighted moving average. It further introduces a unique optimization mechanism that dynamically detects concept drift, quantifies its magnitude, and adjusts the model based on real-time data characteristics. Rigorous evaluations show that it matches batch regression performance in static settings and consistently outperforms or rivals state-of-the-art online models, confirming its effectiveness. Concept drift datasets reveal a performance gap that OLR-WAA effectively bridges, setting it apart from other online models. In addition, the model effectively handles confidence-based scenarios through a conservative update strategy that prioritizes stable, high-confidence data points. Notably, OLR-WAA converges rapidly, consistently yielding higher R2 values compared to other online models.