On Weak-to-Strong Generalization and f-Divergence

TL;DR

This paper introduces the use of $f$-divergence as a loss function in weak-to-strong generalization, providing theoretical insights and empirical evidence of its effectiveness in improving model generalization and noise robustness.

Contribution

It proposes an $f$-divergence framework for W2SG, analyzes its theoretical properties, and demonstrates practical benefits over existing methods.

Findings

$f$-divergence loss improves generalization performance.

Theoretical analysis reveals fundamental limitations and equivalences.

Empirical results show enhanced noise tolerance in models.

Abstract

Weak-to-strong generalization (W2SG) has emerged as a promising paradigm for stimulating the capabilities of strong pre-trained models by leveraging supervision from weaker supervisors. To improve the performance of the strong model, existing methods often require additional weak models or complex procedures, leading to substantial computational and memory overhead. Motivated by the effectiveness of $f$-divergence loss in various machine learning domains, we introduce $f$-divergence as an information-theoretic loss function framework in W2SG. Our theoretical analysis reveals fundamental limitations and equivalence of different $f$-divergence losses in W2SG, supported by sample complexity bounds and information-theoretic insights. We empirically demonstrate that $f$-divergence loss, which generalizes widely-used metrics like KL divergence, effectively improves generalization and noise tolerance of the strong model in practice.