Efficient Multiple Incremental Computation for Kernel Ridge Regression with Bayesian Uncertainty Modeling

TL;DR

This paper introduces an efficient incremental/decremental approach for Kernel Ridge Regression that reduces computational time for big data streams while maintaining accuracy, and extends it to Bayesian uncertainty modeling.

Contribution

It proposes a novel batch processing mechanism for incremental KRR applicable to large-scale and high-dimensional data, including Bayesian uncertainty integration.

Findings

Significantly reduced computational time compared to baseline methods.

Maintained accuracy equivalent to nonincremental approaches.

Effective for variable streaming data analysis.

Abstract

This study presents an efficient incremental/decremental approach for big streams based on Kernel Ridge Regression (KRR), a frequently used data analysis in cloud centers. To avoid reanalyzing the whole dataset whenever sensors receive new training data, typical incremental KRR used a single-instance mechanism for updating an existing system. However, this inevitably increased redundant computational time, not to mention applicability to big streams. To this end, the proposed mechanism supports incremental/decremental processing for both single and multiple samples (i.e., batch processing). A large scale of data can be divided into batches, processed by a machine, without sacrificing the accuracy. Moreover, incremental/decremental analyses in empirical and intrinsic space are also proposed in this study to handle different types of data either with a large number of samples or high feature dimensions, whereas typical methods focused only on one type. At the end of this study, we further the proposed mechanism to statistical Kernelized Bayesian Regression, so that uncertainty modeling with incremental/decremental computation becomes applicable. Experimental results showed that computational time was significantly reduced, better than the original nonincremental design and the typical single incremental method. Furthermore, the accuracy of the proposed method remained the same as the baselines. This implied that the system enhanced efficiency without sacrificing the accuracy. These findings proved that the proposed method was appropriate for variable streaming data analysis, thereby demonstrating the effectiveness of the proposed method.