Learning to Learn: Meta-Critic Networks for Sample Efficient Learning

TL;DR

This paper introduces meta-critic networks that learn to critique and improve learning algorithms, enabling more sample-efficient learning in both reinforcement and supervised settings, especially in few-shot and semi-supervised scenarios.

Contribution

It presents a novel meta-critic framework that generalizes actor-critic methods to learn task-specific critics, including a trainable loss generator for supervised learning.

Findings

Effective in few-shot learning scenarios

Applicable to both reinforcement and supervised learning

Shows promising results on benchmark tasks

Abstract

We propose a novel and flexible approach to meta-learning for learning-to-learn from only a few examples. Our framework is motivated by actor-critic reinforcement learning, but can be applied to both reinforcement and supervised learning. The key idea is to learn a meta-critic: an action-value function neural network that learns to criticise any actor trying to solve any specified task. For supervised learning, this corresponds to the novel idea of a trainable task-parametrised loss generator. This meta-critic approach provides a route to knowledge transfer that can flexibly deal with few-shot and semi-supervised conditions for both reinforcement and supervised learning. Promising results are shown on both reinforcement and supervised learning problems.