Reinforcement Teaching

TL;DR

Reinforcement Teaching introduces a versatile meta-learning framework that trains a teaching policy to enhance any learning algorithm's efficiency, demonstrated through improved reinforcement and supervised learning tasks.

Contribution

The paper presents a unifying meta-learning framework that learns a teaching policy applicable to any algorithm, using a parametric-behavior embedder and progress-based rewards.

Findings

Reinforcement Teaching significantly improves learning performance.

It outperforms heuristic and other parameter-based methods.

The framework is general across different learning paradigms.

Abstract

Machine learning algorithms learn to solve a task, but are unable to improve their ability to learn. Meta-learning methods learn about machine learning algorithms and improve them so that they learn more quickly. However, existing meta-learning methods are either hand-crafted to improve one specific component of an algorithm or only work with differentiable algorithms. We develop a unifying meta-learning framework, called Reinforcement Teaching, to improve the learning process of \emph{any} algorithm. Under Reinforcement Teaching, a teaching policy is learned, through reinforcement, to improve a student's learning algorithm. To learn an effective teaching policy, we introduce the parametric-behavior embedder that learns a representation of the student's learnable parameters from its input/output behavior. We further use learning progress to shape the teacher's reward, allowing it to more quickly maximize the student's performance. To demonstrate the generality of Reinforcement Teaching, we conduct experiments in which a teacher learns to significantly improve both reinforcement and supervised learning algorithms. Reinforcement Teaching outperforms previous work using heuristic reward functions and state representations, as well as other parameter representations.