Rethinking Teacher-Student Curriculum Learning through the Cooperative Mechanics of Experience

TL;DR

This paper introduces a cooperative game theory framework to analyze and improve Teacher-Student Curriculum Learning by understanding how experience composition and order affect learning outcomes.

Contribution

It proposes a data-centric, game-theoretic perspective to interpret TSCL mechanics, enabling better curriculum design and understanding of its effectiveness.

Findings

Equivalent cooperative games can be constructed for TSCL problems.

Game-theoretic analysis reveals key factors influencing curriculum performance.

Value-proportional curricula improve learning even in challenging TSCL scenarios.

Abstract

Teacher-Student Curriculum Learning (TSCL) is a curriculum learning framework that draws inspiration from human cultural transmission and learning. It involves a teacher algorithm shaping the learning process of a learner algorithm by exposing it to controlled experiences. Despite its success, understanding the conditions under which TSCL is effective remains challenging. In this paper, we propose a data-centric perspective to analyze the underlying mechanics of the teacher-student interactions in TSCL. We leverage cooperative game theory to describe how the composition of the set of experiences presented by the teacher to the learner, as well as their order, influences the performance of the curriculum that is found by TSCL approaches. To do so, we demonstrate that for every TSCL problem, an equivalent cooperative game exists, and several key components of the TSCL framework can be reinterpreted using game-theoretic principles. Through experiments covering supervised learning, reinforcement learning, and classical games, we estimate the cooperative values of experiences and use value-proportional curriculum mechanisms to construct curricula, even in cases where TSCL struggles. The framework and experimental setup we present in this work represents a novel foundation for a deeper exploration of TSCL, shedding light on its underlying mechanisms and providing insights into its broader applicability in machine learning.