Online Nonparametric Supervised Learning for Massive Data

TL;DR

This paper introduces an online nonparametric classifier that combines dimension reduction and stochastic approximation to enable real-time supervised learning on massive streaming data, addressing computational challenges of traditional methods.

Contribution

The paper proposes a novel online algorithm for nonparametric classification that efficiently handles high-dimensional, large-scale, and streaming data by integrating PCA and stochastic approximation.

Findings

Online classifier achieves competitive accuracy with batch methods.

Online classifier offers better accuracy/computation trade-off.

Method performs well in real-time fetal monitoring applications.

Abstract

Despite their benefits in terms of simplicity, low computational cost and data requirement, parametric machine learning algorithms, such as linear discriminant analysis, quadratic discriminant analysis or logistic regression, suffer from serious drawbacks including linearity, poor fit of features to the usually imposed normal distribution and high dimensionality. Batch kernel-based nonparametric classifier, which overcomes the linearity and normality of features constraints, represent an interesting alternative for supervised classification problem. However, it suffers from the ``curse of dimension". The problem can be alleviated by the explosive sample size in the era of big data, while large-scale data size presents some challenges in the storage of data and the calculation of the classifier. These challenges make the classical batch nonparametric classifier no longer applicable. This motivates us to develop a fast algorithm adapted to the real-time calculation of the nonparametric classifier in massive as well as streaming data frameworks. This online classifier includes two steps. First, we consider an online principle components analysis to reduce the dimension of the features with a very low computation cost. Then, a stochastic approximation algorithm is deployed to obtain a real-time calculation of the nonparametric classifier. The proposed methods are evaluated and compared to some commonly used machine learning algorithms for real-time fetal well-being monitoring. The study revealed that, in terms of accuracy, the offline (or Batch), as well as, the online classifiers are good competitors to the random forest algorithm. Moreover, we show that the online classifier gives the best trade-off accuracy/computation cost compared to the offline classifier.