Predicting Human Behavior in Unrepeated, Simultaneous-Move Games

TL;DR

This study evaluates models of human behavior in unrepeated, simultaneous-move games, finding that a simplified, modified QLk model best predicts actual human play across multiple data sets.

Contribution

It provides the most comprehensive meta-analysis of five models, introduces a new model family with improved predictive performance and more interpretable parameters.

Findings

QLk model outperforms others in predictive accuracy

Parameters often deviate from economic interpretations

New model family improves parsimony and performance

Abstract

It is common to assume that agents will adopt Nash equilibrium strategies; however, experimental studies have demonstrated that Nash equilibrium is often a poor description of human players' behavior in unrepeated normal-form games. In this paper, we analyze five widely studied models (Quantal Response Equilibrium, Level-k, Cognitive Hierarchy, QLk, and Noisy Introspection) that aim to describe actual, rather than idealized, human behavior in such games. We performed what we believe is the most comprehensive meta-analysis of these models, leveraging ten different data sets from the literature recording human play of two-player games. We began by evaluating the models' generalization or predictive performance, asking how well a model fits unseen test data after having had its parameters calibrated based on separate training data. Surprisingly, we found that what we dub the QLk model of Stahl & Wilson (1994) consistently achieved the best performance. Motivated by this finding, we describe methods for analyzing the posterior distributions over a model's parameters. We found that QLk's parameters were being set to values that were not consistent with their intended economic interpretations. We thus explored variations of QLk, ultimately identifying a new model family that has fewer parameters, gives rise to more parsimonious parameter values, and achieves better predictive performance.