On the Power of Foundation Models

TL;DR

This paper uses category theory to analyze the theoretical capabilities of infinite foundation models, revealing limits of prompt-based learning and the potential of fine tuning for generalization.

Contribution

It introduces a categorical framework to understand foundation models, proving limits of prompt-based learning and showing fine tuning can solve tasks with sufficient resources.

Findings

Prompt-based learning is limited to representable tasks.

Fine tuning can theoretically solve any task within the model’s capacity.

Foundation models can generalize to unseen objects using structural information.

Abstract

With infinitely many high-quality data points, infinite computational power, an infinitely large foundation model with a perfect training algorithm and guaranteed zero generalization error on the pretext task, can the model be used for everything? This question cannot be answered by the existing theory of representation, optimization or generalization, because the issues they mainly investigate are assumed to be nonexistent here. In this paper, we show that category theory provides powerful machinery to answer this question. We have proved three results. The first one limits the power of prompt-based learning, saying that the model can solve a downstream task with prompts if and only if the task is representable. The second one says fine tuning does not have this limit, as a foundation model with the minimum required power (up to symmetry) can theoretically solve downstream tasks for the category defined by pretext task, with fine tuning and enough resources. Our final result can be seen as a new type of generalization theorem, showing that the foundation model can represent unseen objects from the target category (e.g., images) using the structural information from the source category (e.g., texts). Along the way, we provide a categorical framework for supervised and self-supervised learning, which might be of independent interest.