Let's be Honest: An Optimal No-Regret Framework for Zero-Sum Games

TL;DR

This paper introduces a unified framework for solving two-player zero-sum games that achieves optimal regret bounds and removes previous convergence limitations, applicable in both honest and adversarial settings.

Contribution

It presents a novel analysis of optimistic mirror descent, proposes the robust optimistic mirror descent algorithm, and introduces a signaling scheme to combine their advantages.

Findings

Achieves fast convergence for honest regret and game value.

Attains optimal adversarial regret without prior horizon knowledge.

Numerical results show competitive performance of the proposed algorithms.

Abstract

We revisit the problem of solving two-player zero-sum games in the decentralized setting. We propose a simple algorithmic framework that simultaneously achieves the best rates for honest regret as well as adversarial regret, and in addition resolves the open problem of removing the logarithmic terms in convergence to the value of the game. We achieve this goal in three steps. First, we provide a novel analysis of the optimistic mirror descent (OMD), showing that it can be modified to guarantee fast convergence for both honest regret and value of the game, when the players are playing collaboratively. Second, we propose a new algorithm, dubbed as robust optimistic mirror descent (ROMD), which attains optimal adversarial regret without knowing the time horizon beforehand. Finally, we propose a simple signaling scheme, which enables us to bridge OMD and ROMD to achieve the best of both worlds. Numerical examples are presented to support our theoretical claims and show that our non-adaptive ROMD algorithm can be competitive to OMD with adaptive step-size selection.