Machine Teaching for Bayesian Learners in the Exponential Family

TL;DR

This paper introduces an optimal teaching framework for Bayesian learners in the exponential family, formulating it as an optimization problem and providing an approximate algorithm for conjugate models.

Contribution

It presents a novel framework for machine teaching tailored to Bayesian learners and offers an approximate solution for conjugate exponential family models.

Findings

The framework effectively designs training data for Bayesian learners.

An approximate algorithm optimizes teaching sets by focusing on sufficient statistics.

Illustrative examples demonstrate the framework's applicability.

Abstract

What if there is a teacher who knows the learning goal and wants to design good training data for a machine learner? We propose an optimal teaching framework aimed at learners who employ Bayesian models. Our framework is expressed as an optimization problem over teaching examples that balance the future loss of the learner and the effort of the teacher. This optimization problem is in general hard. In the case where the learner employs conjugate exponential family models, we present an approximate algorithm for finding the optimal teaching set. Our algorithm optimizes the aggregate sufficient statistics, then unpacks them into actual teaching examples. We give several examples to illustrate our framework.