A packing lemma for VCN${}_k$-dimension and learning high-dimensional data

TL;DR

This paper extends the theory of high-arity PAC learning by establishing a packing lemma for VCN${}_k$-dimension, completing the characterization of high-arity learnability and linking it to combinatorial dimensions.

Contribution

It proves that non-partite, non-agnostic high-arity PAC learnability implies a high-arity packing property, leading to finiteness of VCN${}_k$-dimension, completing the theoretical framework.

Findings

High-arity PAC learnability implies a high-arity packing property.

Finiteness of VCN${}_k$-dimension is established for certain learnability.

Classic PAC learnability results are extended to high-arity settings.

Abstract

Recently, the authors introduced the theory of high-arity PAC learning, which is well-suited for learning graphs, hypergraphs and relational structures. In the same initial work, the authors proved a high-arity analogue of the Fundamental Theorem of Statistical Learning that almost completely characterizes all notions of high-arity PAC learning in terms of a combinatorial dimension, called the Vapnik--Chervonenkis--Natarajan (VCN${}_k$) $k$-dimension, leaving as an open problem only the characterization of non-partite, non-agnostic high-arity PAC learnability. In this work, we complete this characterization by proving that non-partite non-agnostic high-arity PAC learnability implies a high-arity version of the Haussler packing property, which in turn implies finiteness of VCN${}_k$-dimension. This is done by obtaining direct proofs that classic PAC learnability implies classic Haussler packing property, which in turn implies finite Natarajan dimension and noticing that these direct proofs nicely lift to high-arity.