Capacity dependent analysis for functional online learning algorithms

TL;DR

This paper analyzes the convergence of online stochastic gradient descent algorithms for functional linear models, showing how capacity assumptions can improve convergence rates and differentiate between prediction and estimation tasks.

Contribution

It introduces capacity-dependent analysis that enhances convergence rate understanding and distinguishes prediction from estimation in functional data analysis.

Findings

Capacity assumptions improve convergence rates.

Proper kernel selection can offset regularity requirements.

Prediction and estimation problems behave differently under capacity assumptions.

Abstract

This article provides convergence analysis of online stochastic gradient descent algorithms for functional linear models. Adopting the characterizations of the slope function regularity, the kernel space capacity, and the capacity of the sampling process covariance operator, significant improvement on the convergence rates is achieved. Both prediction problems and estimation problems are studied, where we show that capacity assumption can alleviate the saturation of the convergence rate as the regularity of the target function increases. We show that with properly selected kernel, capacity assumptions can fully compensate for the regularity assumptions for prediction problems (but not for estimation problems). This demonstrates the significant difference between the prediction problems and the estimation problems in functional data analysis.