Online Gradient Descent in Function Space

TL;DR

This paper extends online gradient descent to infinite-dimensional function spaces, providing convergence guarantees and demonstrating its usefulness in various machine learning and optimization problems involving functions or probability densities.

Contribution

It introduces a novel online gradient descent algorithm for Hilbert spaces and analyzes its convergence, bridging online learning with infinite-dimensional optimization.

Findings

Convergence guarantees for the algorithm in Hilbert spaces

Applicability to problems involving probability densities

Potential improvements in streaming function optimization

Abstract

In many problems in machine learning and operations research, we need to optimize a function whose input is a random variable or a probability density function, i.e. to solve optimization problems in an infinite dimensional space. On the other hand, online learning has the advantage of dealing with streaming examples, and better model a changing environ- ment. In this paper, we extend the celebrated online gradient descent algorithm to Hilbert spaces (function spaces), and analyze the convergence guarantee of the algorithm. Finally, we demonstrate that our algorithms can be useful in several important problems.