A Strong Law of Large Numbers for Random Monotone Operators

TL;DR

This paper establishes a strong law of large numbers for random monotone operators, showing that empirical risk minimizers in machine learning converge to the expected risk minimizer as data size grows.

Contribution

It proves a strong law of large numbers for random monotone operators and applies this to demonstrate convergence of empirical risk minimizers in machine learning.

Findings

Proved a strong law of large numbers for random monotone operators.

Showed convergence of empirical risk minimizers to the expected risk minimizer.

Applied the law to stochastic nonsmooth optimization in machine learning.

Abstract

Random monotone operators are stochastic versions of maximal monotone operators which play an important role in stochastic nonsmooth optimization. Several stochastic nonsmooth optimization algorithms have been shown to converge to a zero of a mean operator defined as the expectation, in the sense of the Aumann integral, of a random monotone operator. In this note, we prove a strong law of large numbers for random monotone operators where the limit is the mean operator. We apply this result to the empirical risk minimization problem appearing in machine learning. We show that if the empirical risk minimizers converge as the number of data points goes to infinity, then they converge to an expected risk minimizer.