Ramsey Theorems for Trees and a General 'Private Learning Implies Online Learning' Theorem

TL;DR

This paper proves that in general classification tasks, differentially private learnability guarantees online learnability by establishing new Ramsey-type theorems for trees, extending prior results to broader settings.

Contribution

It introduces novel Ramsey theorems for trees and demonstrates that DP learnability implies online learnability in more general classification scenarios.

Findings

DP learnability implies online learnability for general classification tasks

Established new Ramsey-type theorems for trees

Proof avoids reliance on threshold-based arguments

Abstract

This work continues to investigate the link between differentially private (DP) and online learning. Alon, Livni, Malliaris, and Moran (2019) showed that for binary concept classes, DP learnability of a given class implies that it has a finite Littlestone dimension (equivalently, that it is online learnable). Their proof relies on a model-theoretic result by Hodges (1997), which demonstrates that any binary concept class with a large Littlestone dimension contains a large subclass of thresholds. In a follow-up work, Jung, Kim, and Tewari (2020) extended this proof to multiclass PAC learning with a bounded number of labels. Unfortunately, Hodges's result does not apply in other natural settings such as multiclass PAC learning with an unbounded label space, and PAC learning of partial concept classes. This naturally raises the question of whether DP learnability continues to imply online learnability in more general scenarios: indeed, Alon, Hanneke, Holzman, and Moran (2021) explicitly leave it as an open question in the context of partial concept classes, and the same question is open in the general multiclass setting. In this work, we give a positive answer to these questions showing that for general classification tasks, DP learnability implies online learnability. Our proof reasons directly about Littlestone trees, without relying on thresholds. We achieve this by establishing several Ramsey-type theorems for trees, which might be of independent interest.