Consistency of Extreme Learning Machines and Regression under Non-Stationarity and Dependence for ML-Enhanced Moving Objects

TL;DR

This paper investigates the consistency and asymptotic properties of extreme learning machines and regression methods in non-stationary, dependent spatial-temporal environments, with applications to moving objects collecting data.

Contribution

It provides theoretical guarantees for the consistency and asymptotic normality of least squares, ridge, and LASSO estimators under non-stationary, dependent data settings.

Findings

Consistency of least squares and ridge regression estimates.

Asymptotic normality results for these estimators.

Bounds for sample squared prediction error.

Abstract

Supervised learning by extreme learning machines resp. neural networks with random weights is studied under a non-stationary spatial-temporal sampling design which especially addresses settings where an autonomous object moving in a non-stationary spatial environment collects and analyzes data. The stochastic model especially allows for spatial heterogeneity and weak dependence. As efficient and computationally cheap learning methods (unconstrained) least squares, ridge regression and $\ell_s$-penalized least squares (including the LASSO) are studied. Consistency and asymptotic normality of the least squares and ridge regression estimates as well as corresponding consistency results for the $\ell_s$-penalty are shown under weak conditions. The results also cover bounds for the sample squared predicition error.