Iterative Teaching by Label Synthesis

TL;DR

This paper introduces a label synthesis framework for iterative machine teaching that simplifies example selection by synthesizing labels, achieving exponential teachability with reduced computational cost.

Contribution

The paper proposes a novel label synthesis teaching framework that eliminates the need for exhaustive example selection, providing provable exponential teachability and new algorithms.

Findings

Achieves exponential teachability under the proposed framework

Reduces computational cost compared to traditional methods

Empirically demonstrates effectiveness of the approach

Abstract

In this paper, we consider the problem of iterative machine teaching, where a teacher provides examples sequentially based on the current iterative learner. In contrast to previous methods that have to scan over the entire pool and select teaching examples from it in each iteration, we propose a label synthesis teaching framework where the teacher randomly selects input teaching examples (e.g., images) and then synthesizes suitable outputs (e.g., labels) for them. We show that this framework can avoid costly example selection while still provably achieving exponential teachability. We propose multiple novel teaching algorithms in this framework. Finally, we empirically demonstrate the value of our framework.