Online Monotone Optimization

TL;DR

This paper introduces a novel framework for analyzing no-regret algorithms in dynamic systems, extending online convex optimization to encompass game theory and variational inequalities with a focus on monotonicity.

Contribution

It generalizes online convex optimization to a broader setting, providing a new notion of regret for variational inequalities and demonstrating that monotonicity guarantees no-regret in multi-agent systems.

Findings

A simple update scheme achieves no-regret for monotone systems.

Monotonicity is sufficient for parallel multi-agent no-regret guarantees.

First framework to define regret for variational inequalities.

Abstract

This paper presents a new framework for analyzing and designing no-regret algorithms for dynamic (possibly adversarial) systems. The proposed framework generalizes the popular online convex optimization framework and extends it to its natural limit allowing it to capture a notion of regret that is intuitive for more general problems such as those encountered in game theory and variational inequalities. The framework hinges on a special choice of a system-wide loss function we have developed. Using this framework, we prove that a simple update scheme provides a no-regret algorithm for monotone systems. While previous results in game theory prove individual agents can enjoy unilateral no-regret guarantees, our result proves monotonicity sufficient for guaranteeing no-regret when considering the adjustments of multiple agent strategies in parallel. Furthermore, to our knowledge, this is the first framework to provide a suitable notion of regret for variational inequalities. Most importantly, our proposed framework ensures monotonicity a sufficient condition for employing multiple online learners safely in parallel.