Fairness in Learning: Classic and Contextual Bandits

TL;DR

This paper explores fairness in multi-armed bandit algorithms, proposing fair algorithms with regret bounds, establishing fundamental limits, and connecting fairness with KWIK learning to address both stochastic and contextual bandit problems.

Contribution

It introduces a formal fairness definition for bandits, develops fair algorithms with regret bounds, and links fairness to KWIK learning for contextual bandits, highlighting fundamental trade-offs.

Findings

Fair algorithms have cubic regret dependence on the number of arms.

Any fair algorithm must have at least cubic regret dependence, indicating inherent limitations.

A tight connection between fairness and KWIK learning enables new fair algorithms for linear contextual bandits.

Abstract

We introduce the study of fairness in multi-armed bandit problems. Our fairness definition can be interpreted as demanding that given a pool of applicants (say, for college admission or mortgages), a worse applicant is never favored over a better one, despite a learning algorithm's uncertainty over the true payoffs. We prove results of two types. First, in the important special case of the classic stochastic bandits problem (i.e., in which there are no contexts), we provide a provably fair algorithm based on "chained" confidence intervals, and provide a cumulative regret bound with a cubic dependence on the number of arms. We further show that any fair algorithm must have such a dependence. When combined with regret bounds for standard non-fair algorithms such as UCB, this proves a strong separation between fair and unfair learning, which extends to the general contextual case. In the general contextual case, we prove a tight connection between fairness and the KWIK (Knows What It Knows) learning model: a KWIK algorithm for a class of functions can be transformed into a provably fair contextual bandit algorithm, and conversely any fair contextual bandit algorithm can be transformed into a KWIK learning algorithm. This tight connection allows us to provide a provably fair algorithm for the linear contextual bandit problem with a polynomial dependence on the dimension, and to show (for a different class of functions) a worst-case exponential gap in regret between fair and non-fair learning algorithms