Game Intelligence: Theory and Computation

TL;DR

This paper introduces a formal framework for measuring player intelligence in games using a real-valued score based on strategic ability, mistakes, and reference AI systems, with applications to chess.

Contribution

It proposes the Game Intelligence mechanism and gamingproofness, providing a novel quantitative approach to assess strategic ability in games.

Findings

Magnus Carlsen has the highest GI score among top players.

Stockfish outperforms human players in machine-vs-machine games.

The framework is validated on over a billion chess moves.

Abstract

In this paper, I formalize intelligence measurement in games by introducing mechanisms that assign a real number -- interpreted as an intelligence score -- to each player in a game. This score quantifies the ex-post strategic ability of the players based on empirically observable information, such as the actions of the players, the game's outcome, strength of the players, and a reference oracle machine such as a chess-playing artificial intelligence system. Specifically, I introduce two main concepts: first, the Game Intelligence (GI) mechanism, which quantifies a player's intelligence in a game by considering not only the game's outcome but also the "mistakes" made during the game according to the reference machine's intelligence. Second, I define gamingproofness, a practical and computational concept of strategyproofness. To illustrate the GI mechanism, I apply it to an extensive dataset comprising over a billion chess moves, including over a million moves made by top 20 grandmasters in history. Notably, Magnus Carlsen emerges with the highest GI score among all world championship games included in the dataset. In machine-vs-machine games, the well-known chess engine Stockfish comes out on top.