# Rethinking Formal Models of Partially Observable Multiagent Decision Making

## TL;DR

This paper introduces factored-observation stochastic games (FOSGs) to clarify the relationship between extensive-form games and partially observable stochastic games, enabling better transfer of ideas and improved decomposition in multiagent decision-making.

## Contribution

It proposes FOSGs as a new formalism that connects EFGs and POSGs, and demonstrates how key EFG techniques can be adapted within this framework.

## Key findings

- FOSGs simplify decomposition in multiagent models.
- Unrolling FOSGs yields equivalent extensive-form games.
- Key EFG algorithms are adaptable to FOSGs.

## Abstract

Multiagent decision-making in partially observable environments is usually modelled as either an extensive-form game (EFG) in game theory or a partially observable stochastic game (POSG) in multiagent reinforcement learning (MARL). One issue with the current situation is that while most practical problems can be modelled in both formalisms, the relationship of the two models is unclear, which hinders the transfer of ideas between the two communities. A second issue is that while EFGs have recently seen significant algorithmic progress, their classical formalization is unsuitable for efficient presentation of the underlying ideas, such as those around decomposition. To solve the first issue, we introduce factored-observation stochastic games (FOSGs), a minor modification of the POSG formalism which distinguishes between private and public observation and thereby greatly simplifies decomposition. To remedy the second issue, we show that FOSGs and POSGs are naturally connected to EFGs: by "unrolling" a FOSG into its tree form, we obtain an EFG. Conversely, any perfect-recall timeable EFG corresponds to some underlying FOSG in this manner. Moreover, this relationship justifies several minor modifications to the classical EFG formalization that recently appeared as an implicit response to the model's issues with decomposition. Finally, we illustrate the transfer of ideas between EFGs and MARL by presenting three key EFG techniques -- counterfactual regret minimization, sequence form, and decomposition -- in the FOSG framework.