Sequential Regression Learning with Randomized Algorithms

TL;DR

This paper introduces randomized SINDy, a probabilistic sequential learning algorithm for dynamic data, with proven PAC learning properties and demonstrated effectiveness in regression and classification tasks.

Contribution

It develops a novel randomized, probabilistic approach inspired by SINDy, with rigorous theoretical guarantees and practical updates for dynamic data modeling.

Findings

Proven PAC learning property for the algorithm.

Effective in real-world regression and classification tasks.

Incorporates feature augmentation and regularization for improved performance.

Abstract

This paper presents ``randomized SINDy", a sequential machine learning algorithm designed for dynamic data that has a time-dependent structure. It employs a probabilistic approach, with its PAC learning property rigorously proven through the mathematical theory of functional analysis. The algorithm dynamically predicts using a learned probability distribution of predictors, updating weights via gradient descent and a proximal algorithm to maintain a valid probability density. Inspired by SINDy (Brunton et al. 2016), it incorporates feature augmentation and Tikhonov regularization. For multivariate normal weights, the proximal step is omitted to focus on parameter estimation. The algorithm's effectiveness is demonstrated through experimental results in regression and binary classification using real-world data.