Transitions, Losses, and Re-parameterizations: Elements of Prediction Games

TL;DR

This thesis explores geometric perspectives on various two-player prediction games, providing insights into their inherent challenges and guiding the development of efficient algorithms with strong theoretical guarantees.

Contribution

It offers geometric insights into different prediction games, enhancing understanding of their barriers and informing the design of algorithms with provable guarantees.

Findings

Identifies geometric structures underlying prediction games

Provides bounds on algorithm performance in different game settings

Suggests new approaches for designing efficient learning algorithms

Abstract

This thesis presents some geometric insights into three different types of two player prediction games -- namely general learning task, prediction with expert advice, and online convex optimization. These games differ in the nature of the opponent (stochastic, adversarial, or intermediate), the order of the players' move, and the utility function. The insights shed some light on the understanding of the intrinsic barriers of the prediction problems and the design of computationally efficient learning algorithms with strong theoretical guarantees (such as generalizability, statistical consistency, and constant regret etc.).