Cooperative Meta-Learning with Gradient Augmentation

TL;DR

This paper introduces CML, a cooperative meta-learning framework that enhances gradient-based meta-learning by injecting learnable noise and using a co-learner to improve generalization without additional inference costs.

Contribution

CML is a novel framework that employs a co-learner with gradient augmentation to improve meta-learning performance efficiently.

Findings

CML improves performance in few-shot regression tasks.

CML enhances accuracy in few-shot image classification.

CML is applicable to various gradient-based meta-learning methods.

Abstract

Model agnostic meta-learning (MAML) is one of the most widely used gradient-based meta-learning, consisting of two optimization loops: an inner loop and outer loop. MAML learns the new task from meta-initialization parameters with an inner update and finds the meta-initialization parameters in the outer loop. In general, the injection of noise into the gradient of the model for augmenting the gradient is one of the widely used regularization methods. In this work, we propose a novel cooperative meta-learning framework dubbed CML which leverages gradient-level regularization with gradient augmentation. We inject learnable noise into the gradient of the model for the model generalization. The key idea of CML is introducing the co-learner which has no inner update but the outer loop update to augment gradients for finding better meta-initialization parameters. Since the co-learner does not update in the inner loop, it can be easily deleted after meta-training. Therefore, CML infers with only meta-learner without additional cost and performance degradation. We demonstrate that CML is easily applicable to gradient-based meta-learning methods and CML leads to increased performance in few-shot regression, few-shot image classification and few-shot node classification tasks. Our codes are at https://github.com/JJongyn/CML.