Model Theory and Machine Learning

TL;DR

This paper explores a new connection between model theory and machine learning, linking stability in model theory to learnability in online learning, and providing new examples of learnable classes.

Contribution

It establishes a novel correspondence between stability in model theory and online learnability, expanding the theoretical understanding of learnable classes.

Findings

Identifies a link between stability and online learnability.

Provides new examples of classes that are learnable online.

Highlights parallels between model theory and machine learning concepts.

Abstract

About 25 years ago, it came to light that a single combinatorial property determines both an important dividing line in model theory (NIP) and machine learning (PAC-learnability). The following years saw a fruitful exchange of ideas between PAC learning and the model theory of NIP structures. In this article, we point out a new and similar connection between model theory and machine learning, this time developing a correspondence between \emph{stability} and learnability in various settings of \emph{online learning.} In particular, this gives many new examples of mathematically interesting classes which are learnable in the online setting.