The Sample Complexity of Multi-Distribution Learning for VC Classes

TL;DR

This paper investigates the sample complexity of learning VC classes across multiple distributions, highlighting gaps between existing upper and lower bounds and discussing recent progress and fundamental hurdles in the field.

Contribution

It analyzes the bounds for multi-distribution learning of VC classes, clarifies the gap between known upper and lower bounds, and discusses recent advancements and challenges.

Findings

Upper bound on sample complexity: $O(rac{1}{\

Lower bound: $\\Omega(rac{1}{\

Discussion of fundamental hurdles in game dynamics for statistical learning.

Abstract

Multi-distribution learning is a natural generalization of PAC learning to settings with multiple data distributions. There remains a significant gap between the known upper and lower bounds for PAC-learnable classes. In particular, though we understand the sample complexity of learning a VC dimension d class on $k$ distributions to be $O(\epsilon^{-2} \ln(k)(d + k) + \min\{\epsilon^{-1} dk, \epsilon^{-4} \ln(k) d\})$, the best lower bound is $\Omega(\epsilon^{-2}(d + k \ln(k)))$. We discuss recent progress on this problem and some hurdles that are fundamental to the use of game dynamics in statistical learning.