Model selection for behavioral learning data and applications to contextual bandits

TL;DR

This paper introduces two model selection methods tailored for non-stationary behavioral data, enabling better understanding of individual learning processes and improving contextual bandit applications with theoretical guarantees.

Contribution

It proposes a hold-out procedure and an AIC-type criterion for model selection in non-stationary dependent data, with theoretical error bounds and practical application to human learning data.

Findings

Effective model selection methods for behavioral data

Theoretical error bounds close to i.i.d. case

Successful application to human categorization tasks

Abstract

Learning for animals or humans is the process that leads to behaviors better adapted to the environment. This process highly depends on the individual that learns and is usually observed only through the individual's actions. This article presents ways to use this individual behavioral data to find the model that best explains how the individual learns. We propose two model selection methods: a general hold-out procedure and an AIC-type criterion, both adapted to non-stationary dependent data. We provide theoretical error bounds for these methods that are close to those of the standard i.i.d. case. To compare these approaches, we apply them to contextual bandit models and illustrate their use on both synthetic and experimental learning data in a human categorization task.