Computing the smallest fixed point of order-preserving nonexpansive mappings arising in positive stochastic games and static analysis of programs

TL;DR

This paper characterizes and computes the smallest fixed point of order-preserving nonexpansive maps, with applications in stochastic games and program analysis, introducing a policy iteration algorithm for finite spaces.

Contribution

It provides a novel spectral radius characterization of minimal fixed points and a policy iteration method applicable to finite state and action spaces.

Findings

The policy iteration algorithm finds locally minimal fixed points.

When the discount rate is nonnegative, the fixed point is globally minimal.

The spectral radius condition characterizes fixed point minimality.

Abstract

The problem of computing the smallest fixed point of an order-preserving map arises in the study of zero-sum positive stochastic games. It also arises in static analysis of programs by abstract interpretation. In this context, the discount rate may be negative. We characterize the minimality of a fixed point in terms of the nonlinear spectral radius of a certain semidifferential. We apply this characterization to design a policy iteration algorithm, which applies to the case of finite state and action spaces. The algorithm returns a locally minimal fixed point, which turns out to be globally minimal when the discount rate is nonnegative.