Strategy Synthesis for Zero-Sum Neuro-Symbolic Concurrent Stochastic Games

TL;DR

This paper introduces neuro-symbolic concurrent stochastic games (NS-CSGs), formalizes their properties, and develops practical algorithms for strategy synthesis in continuous-state environments combining neural perception and symbolic decision-making.

Contribution

It proposes a new formalism for neuro-symbolic stochastic games, proves the existence of value functions, and introduces novel value iteration and policy iteration algorithms for continuous state spaces.

Findings

Proved the existence and measurability of the value function for NS-CSGs.

Developed practical VI and PI algorithms for continuous-state models.

Introduced finite abstract representations for strategies and value functions.

Abstract

Neuro-symbolic approaches to artificial intelligence, which combine neural networks with classical symbolic techniques, are growing in prominence, necessitating formal approaches to reason about their correctness. We propose a novel modelling formalism called neuro-symbolic concurrent stochastic games (NS-CSGs), which comprise two probabilistic finite-state agents interacting in a shared continuous-state environment. Each agent observes the environment using a neural perception mechanism, which converts inputs such as images into symbolic percepts, and makes decisions symbolically. We focus on the class of NS-CSGs with Borel state spaces and prove the existence and measurability of the value function for zero-sum discounted cumulative rewards under piecewise-constant restrictions on the components of this class of models. To compute values and synthesise strategies, we present, for the first time, practical value iteration (VI) and policy iteration (PI) algorithms to solve this new subclass of continuous-state CSGs. These require a finite decomposition of the environment induced by the neural perception mechanisms of the agents and rely on finite abstract representations of value functions and strategies closed under VI or PI. First, we introduce a Borel measurable piecewise-constant (B-PWC) representation of value functions, extend minimax backups to this representation and propose a value iteration algorithm called B-PWC VI. Second, we introduce two novel representations for the value functions and strategies, constant-piecewise-linear (CON-PWL) and constant-piecewise-constant (CON-PWC) respectively, and propose Minimax-action-free PI by extending a recent PI method based on alternating player choices for finite state spaces to Borel state spaces, which does not require normal-form games to be solved.