To Reach or not to Reach? Efficient Algorithms for Total-Payoff Games

TL;DR

This paper introduces the first pseudo-polynomial time algorithm for total-payoff games on weighted graphs, using a novel value iteration approach and extending to min-cost reachability games with practical heuristics.

Contribution

It presents a new pseudo-polynomial algorithm for total-payoff games and develops an efficient value iteration method for min-cost reachability games, including heuristics for faster computation.

Findings

First pseudo-polynomial algorithm for total-payoff games

Efficient value iteration algorithm for min-cost reachability games

Heuristics to improve computational speed

Abstract

Quantitative games are two-player zero-sum games played on directed weighted graphs. Total-payoff games (that can be seen as a refinement of the well-studied mean-payoff games) are the variant where the payoff of a play is computed as the sum of the weights. Our aim is to describe the first pseudo-polynomial time algorithm for total-payoff games in the presence of arbitrary weights. It consists of a non-trivial application of the value iteration paradigm. Indeed, it requires to study, as a milestone, a refinement of these games, called min-cost reachability games, where we add a reachability objective to one of the players. For these games, we give an efficient value iteration algorithm to compute the values and optimal strategies (when they exist), that runs in pseudo-polynomial time. We also propose heuristics allowing one to possibly speed up the computations in both cases.