Group-wise oracle-efficient algorithms for online multi-group learning

TL;DR

This paper introduces oracle-efficient algorithms for online multi-group learning that achieve low regret across many overlapping groups without explicitly enumerating them, applicable in various stochastic and adversarial settings.

Contribution

It develops novel algorithms that operate efficiently with large group families via optimization oracles, extending online learning to complex, overlapping group structures.

Findings

Achieves sublinear regret in i.i.d. setting

Extends to adversarial setting with smoothed distributions

Handles adversarial transductive scenarios

Abstract

We study the problem of online multi-group learning, a learning model in which an online learner must simultaneously achieve small prediction regret on a large collection of (possibly overlapping) subsequences corresponding to a family of groups. Groups are subsets of the context space, and in fairness applications, they may correspond to subpopulations defined by expressive functions of demographic attributes. In contrast to previous work on this learning model, we consider scenarios in which the family of groups is too large to explicitly enumerate, and hence we seek algorithms that only access groups via an optimization oracle. In this paper, we design such oracle-efficient algorithms with sublinear regret under a variety of settings, including: (i) the i.i.d. setting, (ii) the adversarial setting with smoothed context distributions, and (iii) the adversarial transductive setting.