Dynamic Structure Embedded Online Multiple-Output Regression for Stream Data

TL;DR

This paper introduces MORES, an online multiple-output regression method that dynamically learns coefficient and residual error structures to improve prediction accuracy and efficiency in streaming data scenarios.

Contribution

The paper presents MORES, a novel online regression algorithm that adaptively models coefficient and residual error structures, enabling fast, accurate predictions on evolving data streams.

Findings

MORES achieves at least 2000 samples/sec processing speed.

It outperforms existing algorithms by over 15 times in speed.

Experimental results validate its effectiveness on real-world datasets.

Abstract

Online multiple-output regression is an important machine learning technique for modeling, predicting, and compressing multi-dimensional correlated data streams. In this paper, we propose a novel online multiple-output regression method, called MORES, for stream data. MORES can \emph{dynamically} learn the structure of the coefficients change in each update step to facilitate the model's continuous refinement. We observe that limited expressive ability of the regression model, especially in the preliminary stage of online update, often leads to the variables in the residual errors being dependent. In light of this point, MORES intends to \emph{dynamically} learn and leverage the structure of the residual errors to improve the prediction accuracy. Moreover, we define three statistical variables to \emph{exactly} represent all the seen samples for \emph{incrementally} calculating prediction loss in each online update round, which can avoid loading all the training data into memory for updating model, and also effectively prevent drastic fluctuation of the model in the presence of noise. Furthermore, we introduce a forgetting factor to set different weights on samples so as to track the data streams' evolving characteristics quickly from the latest samples. Experiments on one synthetic dataset and three real-world datasets validate the effectiveness of the proposed method. In addition, the update speed of MORES is at least 2000 samples processed per second on the three real-world datasets, more than 15 times faster than the state-of-the-art online learning algorithm.