Gaussian Imagination in Bandit Learning

TL;DR

This paper analyzes how Gaussian-based Bayesian agents perform on Bernoulli bandits, showing that with diffuse priors, the regret increase is minimal and diminishes over time, supporting the robustness of Gaussian assumptions.

Contribution

The paper provides theoretical bounds on the regret increase for Gaussian Bayesian agents applied to Bernoulli bandits, formalizing the robustness of Gaussian assumptions in misspecified settings.

Findings

Regret increase grows at most as the square root of time horizon

Per-timestep regret increase vanishes with diffuse priors

Gaussian agents remain effective under misspecification

Abstract

Assuming distributions are Gaussian often facilitates computations that are otherwise intractable. We study the performance of an agent that attains a bounded information ratio with respect to a bandit environment with a Gaussian prior distribution and a Gaussian likelihood function when applied instead to a Bernoulli bandit. Relative to an information-theoretic bound on the Bayesian regret the agent would incur when interacting with the Gaussian bandit, we bound the increase in regret when the agent interacts with the Bernoulli bandit. If the Gaussian prior distribution and likelihood function are sufficiently diffuse, this increase grows at a rate which is at most linear in the square-root of the time horizon, and thus the per-timestep increase vanishes. Our results formalize the folklore that so-called Bayesian agents remain effective when instantiated with diffuse misspecified distributions.