Accelerated Multiplicative Weights Update Avoids Saddle Points almost always

TL;DR

This paper introduces an accelerated version of the Multiplicative Weights Update algorithm, based on Riemannian Accelerated Gradient Descent, which provably almost always avoids saddle points in non-convex optimization on product simplices.

Contribution

It proposes a novel accelerated MWU algorithm using Riemannian geometry and proves its effectiveness in avoiding saddle points almost always.

Findings

Accelerated MWU almost always avoids saddle points.

The method is based on Riemannian Accelerated Gradient Descent.

The approach extends MWU's applicability in non-convex optimization.

Abstract

We consider non-convex optimization problems with constraint that is a product of simplices. A commonly used algorithm in solving this type of problem is the Multiplicative Weights Update (MWU), an algorithm that is widely used in game theory, machine learning and multi-agent systems. Despite it has been known that MWU avoids saddle points, there is a question that remains unaddressed:"Is there an accelerated version of MWU that avoids saddle points provably?" In this paper we provide a positive answer to above question. We provide an accelerated MWU based on Riemannian Accelerated Gradient Descent, and prove that the Riemannian Accelerated Gradient Descent, thus the accelerated MWU, almost always avoid saddle points.