Non-Euclidean Monotone Operator Theory and Applications

TL;DR

This paper extends monotone operator theory from Hilbert spaces to finite-dimensional spaces with non-Euclidean norms, enabling new algorithms and bounds for machine learning and optimization problems.

Contribution

It introduces a generalization of monotone operator theory to non-Euclidean spaces using weak pairings and logarithmic norms, and demonstrates convergence of classical methods in this setting.

Findings

Operators exhibit similar properties to Hilbert space case

Classical iterative methods converge in non-Euclidean spaces

New bounds for neural network Lipschitz constants

Abstract

While monotone operator theory is often studied on Hilbert spaces, many interesting problems in machine learning and optimization arise naturally in finite-dimensional vector spaces endowed with non-Euclidean norms, such as diagonally-weighted $\ell_{1}$ or $\ell_{\infty}$ norms. This paper provides a natural generalization of monotone operator theory to finite-dimensional non-Euclidean spaces. The key tools are weak pairings and logarithmic norms. We show that the resolvent and reflected resolvent operators of non-Euclidean monotone mappings exhibit similar properties to their counterparts in Hilbert spaces. Furthermore, classical iterative methods and splitting methods for finding zeros of monotone operators are shown to converge in the non-Euclidean case. We apply our theory to equilibrium computation and Lipschitz constant estimation of recurrent neural networks, obtaining novel iterations and tighter upper bounds via forward-backward splitting.