Modeling Uncertainty: Constraint-Based Belief States in Imperfect-Information Games

TL;DR

This paper explores constraint-based and probabilistic methods for representing beliefs in imperfect-information games, showing that simpler constraint-based models perform comparably to probabilistic inference in agent decision-making.

Contribution

It introduces a constraint-based belief state model for imperfect-information games and compares it with probabilistic methods, demonstrating comparable effectiveness.

Findings

Constraint-based beliefs perform similarly to probabilistic inference.

Minimal performance difference between the two belief representations.

Constraint-based approach simplifies belief modeling without sacrificing effectiveness.

Abstract

In imperfect-information games, agents must make decisions based on partial knowledge of the game state. The Belief Stochastic Game model addresses this challenge by delegating state estimation to the game model itself. This allows agents to operate on externally provided belief states, thereby reducing the need for game-specific inference logic. This paper investigates two approaches to represent beliefs in games with hidden piece identities: a constraint-based model using Constraint Satisfaction Problems and a probabilistic extension using Belief Propagation to estimate marginal probabilities. We evaluated the impact of both representations using general-purpose agents across two different games. Our findings indicate that constraint-based beliefs yield results comparable to those of probabilistic inference, with minimal differences in agent performance. This suggests that constraint-based belief states alone may suffice for effective decision-making in many settings.