Representations Shape Weak-to-Strong Generalization: Theoretical Insights and Empirical Predictions

TL;DR

This paper offers a theoretical framework and empirical evidence showing how weak supervision can guide stronger models, with kernels derived from internal representations predicting performance trends across various tasks.

Contribution

It introduces a kernel-based theoretical characterization of weak-to-strong generalization, linking internal representations to performance prediction without labels.

Findings

Kernel-based metrics predict W2SG performance trends.

Strong models can surpass weak supervisors even with imperfect supervision.

Representation analysis explains error correction in weak supervision.

Abstract

Weak-to-Strong Generalization (W2SG), where a weak model supervises a stronger one, serves as an important analogy for understanding how humans might guide superhuman intelligence in the future. Promising empirical results revealed that a strong model can surpass its weak supervisor. While recent work has offered theoretical insights into this phenomenon, a clear understanding of the interactions between weak and strong models that drive W2SG remains elusive. We investigate W2SG through a theoretical lens and show that it can be characterized using kernels derived from the principal components of weak and strong models' internal representations. These kernels can be used to define a space that, at a high level, captures what the weak model is unable to learn but is learnable by the strong model. The projection of labels onto this space quantifies how much the strong model falls short of its full potential due to weak supervision. This characterization also provides insights into how certain errors in weak supervision can be corrected by the strong model, regardless of overfitting. Our theory has significant practical implications, providing a representation-based metric that predicts W2SG performance trends without requiring labels, as shown in experiments on molecular predictions with transformers and 5 NLP tasks involving 52 LLMs.