Refined Analysis of FPL for Adversarial Markov Decision Processes

TL;DR

This paper improves the theoretical analysis of Follow-the-Perturbed-Leader algorithms for adversarial Markov Decision Processes, achieving regret bounds comparable to the best existing algorithms with simpler methods.

Contribution

It provides a refined analysis of FPL algorithms in adversarial MDPs, matching the best regret bounds with faster, simpler algorithms.

Findings

Achieves regret bounds matching the best known results.

Introduces faster and simpler FPL-based algorithms.

Provides improved theoretical guarantees for adversarial MDPs.

Abstract

We consider the adversarial Markov Decision Process (MDP) problem, where the rewards for the MDP can be adversarially chosen, and the transition function can be either known or unknown. In both settings, Follow-the-PerturbedLeader (FPL) based algorithms have been proposed in previous literature. However, the established regret bounds for FPL based algorithms are worse than algorithms based on mirrordescent. We improve the analysis of FPL based algorithms in both settings, matching the current best regret bounds using faster and simpler algorithms.