Continuous-time Online Learning via Mean-Field Neural Networks: Regret Analysis in Diffusion Environments

TL;DR

This paper analyzes continuous-time online learning with neural networks in diffusion environments, providing regret bounds and demonstrating the method's effectiveness through simulations.

Contribution

It introduces a regret analysis framework for neural network-based online learning in diffusion settings, using mean-field limits and advanced mathematical tools.

Findings

Constant static regret bound under displacement convexity

Explicit linear regret bounds in non-convex settings

Simulations show the approach outperforms traditional methods

Abstract

We study continuous-time online learning where data are generated by a diffusion process with unknown coefficients. The learner employs a two-layer neural network, continuously updating its parameters in a non-anticipative manner. The mean-field limit of the learning dynamics corresponds to a stochastic Wasserstein gradient flow adapted to the data filtration. We establish regret bounds for both the mean-field limit and finite-particle system. Our analysis leverages the logarithmic Sobolev inequality, Polyak-Lojasiewicz condition, Malliavin calculus, and uniform-in-time propagation of chaos. Under displacement convexity, we obtain a constant static regret bound. In the general non-convex setting, we derive explicit linear regret bounds characterizing the effects of data variation, entropic exploration, and quadratic regularization. Finally, our simulations demonstrate the outperformance of the online approach and the impact of network width and regularization parameters.