Computing Fixpoints of Learned Functions: Chaotic Iteration and Simple Stochastic Games

TL;DR

This paper introduces a generalized dampened Mann iteration scheme that enhances fixpoint computation for approximate functions, enabling chaotic iterations and application to probabilistic models like simple stochastic games.

Contribution

It generalizes the dampened Mann iteration to allow flexible learning rates, facilitating higher-dimensional fixpoint computations and applications to stochastic games.

Findings

The new scheme relaxes previous convergence constraints.

It enables chaotic iterations for complex, high-dimensional problems.

It applies directly to compute expected payoffs in probabilistic models.

Abstract

The problem of determining the (least) fixpoint of (higher-dimensional) functions over the non-negative reals frequently occurs when dealing with systems endowed with a quantitative semantics. We focus on the situation in which the functions of interest are not known precisely but can only be approximated. As a first contribution we generalize an iteration scheme called dampened Mann iteration, recently introduced in the literature. The improved scheme relaxes previous constraints on parameter sequences, allowing learning rates to converge to zero or not converge at all. While seemingly minor, this flexibility is essential to enable the implementation of chaotic iterations, where only a subset of components is updated in each step, allowing to tackle higher-dimensional problems. Additionally, by allowing learning rates to converge to zero, we can relax conditions on the convergence speed of function approximations, making the method more adaptable to various scenarios. We also show that dampened Mann iteration applies immediately to compute the expected payoff in various probabilistic models, including simple stochastic games, not covered by previous work.