Conformal Bandits: Bringing statistical validity and reward efficiency to the small-gap regime

TL;DR

Conformal Bandits integrate conformal prediction into bandit algorithms to provide finite-sample statistical guarantees and improve reward efficiency in small-gap regimes, especially in financial portfolio applications.

Contribution

This paper introduces Conformal Bandits, a novel framework that combines conformal prediction with bandit algorithms to ensure statistical validity and enhance performance in small-gap scenarios.

Findings

Conformal Bandits achieve better regret in small-gap regimes.

They provide finite-sample coverage guarantees unlike classical policies.

Enhanced exploration-exploitation with hidden Markov models improves financial returns.

Abstract

We introduce Conformal Bandits, a novel framework integrating Conformal Prediction (CP) into bandit problems, a classic paradigm for sequential decision-making under uncertainty. Traditional regret-minimisation bandit strategies like Thompson Sampling and Upper Confidence Bound (UCB) typically rely on distributional assumptions or asymptotic guarantees; further, they remain largely focused on regret, neglecting their statistical properties. We address this gap. Through the adoption of CP, we bridge the regret-minimising potential of a decision-making bandit policy with statistical guarantees in the form of finite-time prediction coverage. We demonstrate the potential of it Conformal Bandits through simulation studies and an application to portfolio allocation, a typical small-gap regime, where differences in arm rewards are far too small for classical policies to achieve optimal regret bounds in finite sample. Motivated by this, we showcase our framework's practical advantage in terms of regret in small-gap settings, as well as its added value in achieving nominal coverage guarantees where classical UCB policies fail. Focusing on our application of interest, we further illustrate how integrating hidden Markov models to capture the regime-switching behaviour of financial markets, enhances the exploration-exploitation trade-off, and translates into higher risk-adjusted regret efficiency returns, while preserving coverage guarantees.