Limit-Computable Grains of Truth for Arbitrary Computable Extensive-Form (Un)Known Games

TL;DR

This paper introduces a broad class of strategies for infinite multi-player games that includes all computable and Bayes-optimal strategies, providing convergence guarantees and practical approximations in known and unknown environments.

Contribution

It offers a formal, general solution to the grain of truth problem, encompassing all computable strategies and ensuring convergence in various game settings.

Findings

Convergence to equilibrium in known repeated games.

Thompson sampling leads to ε-Nash equilibria in unknown environments.

Strategies can be approximated arbitrarily closely computationally.

Abstract

A Bayesian player acting in an infinite multi-player game learns to predict the other players' strategies if his prior assigns positive probability to their play (or contains a grain of truth). Kalai and Lehrer's classic grain of truth problem is to find a reasonably large class of strategies that contains the Bayes-optimal policies with respect to this class, allowing mutually-consistent beliefs about strategy choice that obey the rules of Bayesian inference. Only small classes are known to have a grain of truth and the literature contains several related impossibility results. In this paper we present a formal and general solution to the full grain of truth problem: we construct a class of strategies wide enough to contain all computable strategies as well as Bayes-optimal strategies for every reasonable prior over the class. When the "environment" is a known repeated stage game, we show convergence in the sense of [KL93a] and [KL93b]. When the environment is unknown, agents using Thompson sampling converge to play $\varepsilon$-Nash equilibria in arbitrary unknown computable multi-agent environments. Finally, we include an application to self-predictive policies that avoid planning. While these results use computability theory only as a conceptual tool to solve a classic game theory problem, we show that our solution can naturally be computationally approximated arbitrarily closely.