Learning to Forget for Meta-Learning

TL;DR

This paper introduces L2F, a method that dynamically attenuates shared initializations in meta-learning to reduce task conflicts, enabling faster adaptation and improved performance in few-shot learning scenarios.

Contribution

The paper proposes a novel task-and-layer-wise attenuation technique called L2F that enhances MAML by reducing conflicting influences of prior knowledge, improving adaptation speed and accuracy.

Findings

L2F accelerates task adaptation in few-shot learning.

L2F improves performance of existing MAML-based frameworks.

L2F is simple to implement and broadly applicable.

Abstract

Few-shot learning is a challenging problem where the goal is to achieve generalization from only few examples. Model-agnostic meta-learning (MAML) tackles the problem by formulating prior knowledge as a common initialization across tasks, which is then used to quickly adapt to unseen tasks. However, forcibly sharing an initialization can lead to conflicts among tasks and the compromised (undesired by tasks) location on optimization landscape, thereby hindering the task adaptation. Further, we observe that the degree of conflict differs among not only tasks but also layers of a neural network. Thus, we propose task-and-layer-wise attenuation on the compromised initialization to reduce its influence. As the attenuation dynamically controls (or selectively forgets) the influence of prior knowledge for a given task and each layer, we name our method as L2F (Learn to Forget). The experimental results demonstrate that the proposed method provides faster adaptation and greatly improves the performance. Furthermore, L2F can be easily applied and improve other state-of-the-art MAML-based frameworks, illustrating its simplicity and generalizability.

Figures

Tables

Table 1. Table 1: Test accuracy on 5-way miniImageNet classification

	Backbone	miniImageNet
		1-shot	5-shot
Matching Network [35]	4 conv	$43.44\pm 0.77\%$	$55.31\pm 0.73\%$
Meta-Learner LSTM [21]	4 conv	$43.56\pm 0.84\%$	$60.60\pm 0.71\%$
MetaNet [17]	5 conv	$49.21\pm 0.96\%$	$-$
LLAMA [6]	4 conv	$49.40\pm 0.84\%$	$-$
Relation Network [31]	4 conv	$50.44\pm 0.82\%$	$65.32\pm 0.70\%$
Prototypical Network [30]	4 conv	$49.42\pm 0.78\%$	$68.20\pm 0.66\%$
\hdashlineMAML [5]	4 conv	$48.70\pm 1.75\%$	$63.11\pm 0.91\%$
MAML++ [1]	4 conv	$52.15\pm 0.26\%$	$68.32\pm 0.44\%$
MAML+L2F (Ours)	4 conv	$52.10\pm 0.50\%$	$69.38\pm 0.46\%$
MetaGAN [38]	ResNet12	$52.71\pm 0.64\%$	$68.63\pm 0.67\%$
SNAIL [16]	ResNet12*	$55.71\pm 0.99\%$	$68.88\pm 0.92\%$
adaResNet [18]	ResNet12	$56.88\pm 0.62\%$	$71.94\pm 0.57\%$
CAML [9]	ResNet12*	$59.23\pm 0.99\%$	$72.35\pm 0.71\%$
TADAM [20]	ResNet12*	$58.5\pm 0.3\%$	$76.7\pm 0.3\%$
\hdashlineMAML	ResNet12	$51.03\pm 0.50\%$	$68.26\pm 0.47\%$
MAML+L2F (Ours)	ResNet12	$57.48\pm 0.49\%$	$74.68\pm 0.43\%$
LEO [24]	WRN-28-10*	$61.76\pm 0.08\%$	$77.59\pm 0.12\%$
LEO (reproduced)	WRN-28-10*	$61.50\pm 0.17\%$	$77.12\pm 0.07\%$
LEO+L2F (Ours)	WRN-28-10*	$62.12\pm 0.13\%$	$78.13\pm 0.15\%$

Table 2. Table 2: Test accuracy on 5-way tieredImageNet classification

	Backbone	tieredImageNet
		1-shot	5-shot
MAML	4 conv	$49.06\pm 0.50\%$	$67.48\pm 0.47\%$
MAML+L2F (Ours)	4 conv	$54.40\pm 0.50\%$	$73.34\pm 0.44\%$
MAML	ResNet12	$58.58\pm 0.49\%$	$71.24\pm 0.43\%$
MAML+L2F (Ours)	ResNet12	$63.94\pm 0.48\%$	$77.61\pm 0.41\%$
LEO	WRN-28-10*	$66.33\pm 0.05\%$	$81.44\pm 0.09\%$
LEO (reproduced)	WRN-28-10*	$67.02\pm 0.11\%$	$82.29\pm 0.16\%$
LEO+L2F (Ours)	WRN-28-10*	$68.00\pm 0.11\%$	$83.02\pm 0.08\%$

Table 3. Table 3: Ablation studies on inner-loop update steps on 5-way 5-shot miniImageNet classification.

Inner-loop	MAML	MAML+L2F(Ours)
update steps
1	$56.93\pm 0.32\%$	$68.16\pm 0.47\%$
2	$55.63\pm 0.50\%$	$66.85\pm 0.49\%$
3	$58.79\pm 0.49\%$	$68.61\pm 0.46\%$
4	$62.72\pm 0.45\%$	$68.66\pm 0.43\%$
5	$63.94\pm 0.41\%$	$69.38\pm 0.46\%$
6	$64.54\pm 0.46\%$	$-$

Table 4. Table 4: Ablation studies on attenuation scope. Except MAML+L2F, all models learn task-independent attenuation parameters to illustrate the effect of attenuation scope alone, without task-conditioning.

Attenuation Scope	Accuracy
None (MAML, our reproduction)	$63.94\pm 0.48\%$
parameter-wise	$64.7\pm 0.43\%$
filter-wise	$65.35\pm 0.48\%$
layer-wise	$68.49\pm 0.41\%$
network-wise	$67.84\pm 0.46\%$
MAML+L2F (Ours)	$69.38\pm 0.46\%$

Table 5. Table 5: Ablation studies on types of representation for task embedding

	miniImageNet
	5-shot
Features (class prototype)	$68.73\pm 0.46\%$
Gradients (Ours, MAML+L2F)	$69.48\pm 0.46\%$

Table 6. Table 6: Ablation studies on task-conditioned transformation to illustrate the effectiveness of attenuation.

Model	Description	Accuracy
1	MAML (our reproduction)	$63.94\pm 0.48\%$
2	MAML + task-dependent non-sigmoided ${\gamma }_i^j,{\delta }_i^j$	$66.22\pm 0.47\%$
3	MAML + task-dependent non-sigmoided ${\gamma }_i^j$	$67.56\pm 0.47\%$
Ours	MAML + L2F (task-dependent sigmoided ${\gamma }_i^j$)	$69.38\pm 0.46\%$

Table 7. Table 7: MSE averaged over the sampled 100 points with 95 % percent \% confidence intervals on k 𝑘 k -shot regression. Our method consistently outperforms across all gradient steps.

	Models	1 step	2 steps	5 steps
5-shot training	MAML	1.2247	1.0268	0.8995
	MAML+L2F (Ours)	1.0537	0.8426	0.7096
10-shot training	MAML	0.9884	0.6192	0.4072
	MAML+L2F (Ours)	0.8069	0.5317	0.3696
20-shot training	MAML	0.6144	0.3346	0.1817
	MAML+L2F (Ours)	0.5475	0.2805	0.1629

Table 8. Table H: Test accuracy on FC100 5-way classification

	FC100
	1-shot	5-shot	10-shot
MAML*	$35.98\pm 0.48\%$	$51.40\pm 0.50\%$	$56.13\pm 0.50\%$
MAML+L2F	$39.46\pm 0.49\%$	$53.12\pm 0.50\%$	$59.72\pm 0.49\%$

Table 9. Table I: Test accuracy on CIFAR-FS 5-way classification

	CIFAR-FS
	1-shot	5-shot
MAML*	$53.91\pm 0.50\%$	$70.16\pm 0.46\%$
MAML+L2F	$57.28\pm 0.49\%$	$73.94\pm 0.44\%$

Table 10. Table J: Test accuracy (%) of MAML (our reproduction) vs MAML+L2F on Meta-Dataset

Model	ILSVRC	Omniglot	Aircraft	Birds	Textures	Quick Draw	Fungi	VGG Flower	Traffic signs	MSCOCO
MAML	$19.35\pm 0.84$	$66.14\pm 1.47$	$40.20\pm 1.05$	$40.61\pm 1.79$	$38.94\pm 1.69$	$42.46\pm 1.54$	$13.80\pm 1.19$	$61.07\pm 1.50$	$23.38\pm 1.12$	$13.29\pm 1.11$
Ours	$25.93\pm 1.10$	$72.26\pm 1.63$	$53.31\pm 1.48$	$42.62\pm 1.30$	$49.57\pm 1.03$	$50.28\pm 1.67$	$20.20\pm 1.08$	$64.23\pm 1.31$	$31.71\pm 1.45$	$19.75\pm 0.93$

Table 11. (a) Regression

	Models	1 step	2 steps	5 steps
5-shot	MAML	1.2247	1.0268	0.8995
	MuMoMAML	1.1010	0.9291	0.8615
	MAML++	1.2028	0.9268	0.7547
	Ours	1.0537	0.8426	0.7096

Table 12. (a) Regression

	Models	1 step	2 steps	5 steps
5-shot	MAML	1.2247	1.0268	0.8995
	MuMoMAML	1.1010	0.9291	0.8615
	MAML++	1.2028	0.9268	0.7547
	Ours	1.0537	0.8426	0.7096

Table 13. (b) RL

	Models	1 step	2 steps	3 steps
2D Navi	MAML	-32.626	-25.746	-20.734
	MuMoMAML	-25.785	-23.705	-19.747
	MAML++	-36.281	-27.264	-18.620
	Ours	-24.230	-19.598	-16.517

Table 14. (a) Regression

	Hyperparameters
policy network	2 hidden layers of size 40 with ReLU
training iterations	50000 epochs
inner update $\alpha$	0.01
meta update optimizer	Adam
meta batch size	4
k shot	[5,10,20]
loss function	MSE loss
eval	randomly sample 100 sine curves,
	sample 100 examples (repeated 100 times)

Table 15. (a) Regression

	Hyperparameters
policy network	2 hidden layers of size 40 with ReLU
training iterations	50000 epochs
inner update $\alpha$	0.01
meta update optimizer	Adam
meta batch size	4
k shot	[5,10,20]
loss function	MSE loss
eval	randomly sample 100 sine curves,
	sample 100 examples (repeated 100 times)

Table 16. (b) RL

	Hyperparameters
policy network	2 hidden layers of size 100 with ReLU
inner update	vanilla policy gradient
inner update $\alpha$	0.01
meta-optimizer	TRPO
training iterations	500 epochs, choose best model
MuJoCo horizon	200
MuJoCo batch size	40
MUJoCo evals	update 4 gradient updates,
	each with 40 samples for a task
2D navigation horizon	100
2D navigation batch size	20
2D navigation evals	update 4 gradient updates,
	each with 20 samples for a task

Equations

$${\theta }_i^{\prime }=\theta -\alpha {\nabla }_{\theta }{\mathcal{L}}_{{\mathcal{T}}_i}^{{\mathcal{D}}_{{\mathcal{T}}_i}}(f_{\theta }).$$

$$\theta \leftarrow \theta -\eta {\nabla }_{\theta }\sum _{{\mathcal{T}}_i}{\mathcal{L}}_{{\mathcal{T}}_i}^{{\mathcal{D}}_{{\mathcal{T}}_i}^{{}^{\prime }}}(f_{{\theta }_i^{\prime }}).$$

$$\overline{{\theta }^j}={\gamma }^j{\theta }^j,$$

$${\gamma }_i=g_{\varphi }({\nabla }_{\theta }{\mathcal{L}}_{{\mathcal{T}}_i}^{{\mathcal{D}}_{{\mathcal{T}}_i}}(f_{\theta })),$$

Full text

1 Introduction

Recent deep learning models demonstrate outstanding performance in various fields; however, they require supervised learning with a tremendous amount of labeled data. On the other hand, humans are able to learn concepts from only few examples. Considering the cost of data annotation, the capability of humans to learn from few examples is desirable.

When there are concerns for overfitting in few-data regime, data augmentation and regularization techniques are often used. Another commonly used technique is to fine-tune a network pre-trained on large labelled data from another dataset or task [22, 29]. Fine-tuning often does provide adaptation without overfitting even in few-data regime, however at the cost of computation due to many update iterations [34]. In contrast, meta-learning tackles the problem systematically via two stages of learners: a meta-learner learns common knowledge across a distribution of tasks, which is then used for a learner to quickly learn task-specific knowledge with few examples. A popular instance is the model-agnostic meta-learning (MAML) [5], where a meta-learner is formulated such that it learns a common initialization that encodes the common knowledge across tasks.

The assumption of the existence of a task distribution may justify MAML for seeking a common initialization among tasks. But, there still exists variations among tasks, some of which may lead to the disagreement among tasks on the location of the initialization. We call such disagreement conflict and formally define it in this paper. Some of prior knowledge encoded in such compromised initialization is useful for one task but may be irrelevant or even detrimental for another. Consequently, a learner struggles to learn new concepts quickly with the prior knowledge that conflicts with information from new examples, as illustrated in Figure  1. Such learning difficulty can manifest as the sharp loss landscape and thereby poor generalization to new examples [13, 26]. Motivated by our hypothesis, we analyze and indeed observe the sharp landscape during fast adaptation to new examples (as shown in Figure 2) and suggest that the learned initialization by MAML is a “bad” location.

One solution for a meta-learner would be to simply forget the part of the initialization that hinders adaptation to the task, minimizing its influence. This raises two questions: Where do these conflicts occur? To what extent? We hypothesize that the degree of conflict varies among layers of a neural network, especially CNN, since deeper layers learn more task-specific knowledge or class-specific knowledge in classification [37]. To test the hypothesis, we measure conflict at each layer and observe that conflict is indeed more severe at deeper layers, as shown in Figure 3LABEL:sub@fig:layer_conflict. We also observe that the amount of agreement between the learned initialization and the initialization desired by a given task differs for each task in Figure 3LABEL:sub@fig:task_conflict. Thus, we argue that conflicts occur at two levels: task and layer.

Motivated by the observation, we propose to learn selective forgetting by applying a task-and-layer-wise attenuation on MAML initialization, controlling the influence of prior knowledge for each task and layer. For each task, we argue that initialization weights and its gradients (obtained from support examples of task), together, encode information about optimization specific to a task, and thus propose to condition on them to generate attenuation parameters. As for layer-wise attenuation, we generate an attenuation parameter for each layer. The proposed method, named L2F (Learn to Forget), indeed improves the quality of the initialization (illustrated by a smoother loss landscape in Figure 2) and consistent performance improvement across different domains, managing to maintain the simplicity and generalizability of MAML.

2 Related Work

Meta-learning aims to learn across-task prior knowledge to achieve fast adaptation to specific tasks [2, 7, 27, 28, 32]. Recent meta-learning systems can be broadly classified into three categories: metric-based, network-based, and optimization-based. The goal of metric-based system is to learn relationship between query and support examples by learning an embedding space, where similar classes are closer and different classes are further apart [10, 30, 31, 35]. Network-based approaches encode fast adaptation into network architecture, for example, by generating input-conditioned weights [17, 20] or employing an external memory [18, 25]. On the other hand, optimization-based systems adjust optimization for fast adaptation [5, 21, 19].

Among optimization-based systems, MAML [5] has recently received interests, owing to its simplicity and generalizability. The generalizability stems from its model-agnostic algorithm that learns across-task initialization. The initialization aims to encode prior knowledge that helps the model quickly learn and achieve good generalization performance over tasks on average. While MAML boasts the simplicity, it shows relatively low performance on few-shot learning.

There has been several works that tried to improve the performance, especially on few-shot classification [1, 12, 14, 38, 9]. However, none of these methods tackles the problem with the sharing of the starting point of adaptation to different tasks. Recently, there has been a few works [20, 24, 36] that try to achieve task-wise model or initialization through their proposed task embeddings. The metric-based system has a similar issue with MAML, and thus TADAM [20] proposes to learn task embeddings, which are then used to generate affine transformation parameters that transform the features.

In this work, we focus on analyzing the problems of MAML and improving its performance, while maintaining its generalizability. LEO [24] tries to solve the issue with the shared initialization by learning task embeddings through relation network, which are then used to generate input-dependent initializations in low-dimensional latent space. Another work that tries to relax the constraint on sharing the initialization is Multimodal MAML [36], where they propose to learn task embeddings and transform the MAML initialization with affine parameters.

In contrast to [36, 24] that only focus on making the initialization task-dependent, we approach the problem from the perspective of optimization and provide a new insight that the quality of MAML initialization is compromised due to conflicts among tasks on the location of the initialization in optimization landscape. Such compromised initialization will hinder fast adaptation and is illustrated by sharp loss landscape in Figure 2. Motivated by the phenomenon of conflicts, we argue that we only need to attenuate (forget) the compromised part of the initialization. In fact, a large portion of the performance boost comes from the attenuation, not from the task-conditioned transformation (see Table 4).

From the perspective of optimization, we also provide more effective and efficient task embedding. Previous works [24, 36] try to achieve task-wise initializations through learning task embeddings directly from the input. However, learning such task embeddings without any task label is difficult and require specialized techniques, such as relation network [24] and metric learning [20] that may not be applicable in other complex problems such as in reinforcement learning. We argue and observe that the amount of conflicts varies among tasks, hinting that conflicts can be used to identify tasks. Since conflicts between the desired initialization by task and the learned initialization can be described with gradients (see Section 3.3), we demonstrate that gradients itself give task-specific optimization information and thus can be used to represent tasks. Because gradients are easily obtainable and model-agnostic, not only do we achieve effective task-wise initialization but also manage to maintain the simplicity and generalizability of MAML.

Overall, our proposed method greatly improves the performance of MAML while managing to maintain the simplicity and generalizability of MAML. Owing to its generalizability, we further show that not only does our method demonstrate a consistent improvement across domains, including reinforcement learning; but also our method can be easily applied to other MAML-based methods.

3 Proposed Method

3.1 Problem Formulation

Before introducing the proposed method, we start with the formulation of a generic meta-learning algorithm. We assume there is a distribution of tasks $p(\mathcal{T})$, from which meta-learning algorithm aims to learn the prior knowledge, represented by a model with parameters $\theta$. Tasks, each of which is sampled from $p(\mathcal{T})$, are split into three disjoint sets: meta-training set, meta-validation set, and meta-test set. In $k$-shot learning, a task $\mathcal{T}_{i}$ is first sampled from the meta-training set, followed by sampling $k$ number of examples $\mathcal{D}_{\mathcal{T}_{i}}$ from $\mathcal{T}_{i}$. These $k$ examples are then used to quickly adapt a model with parameters, $\theta$. Then, new examples $\mathcal{D}^{\mathcal{{}^{\prime}}}_{\mathcal{T}_{i}}$ are sampled from the same task $\mathcal{T}_{i}$ to evaluate the generalization performance on unseen examples with the corresponding loss function, $\mathcal{L}_{\mathcal{T}_{i}}$. The feedback from the loss is then used to adjust the model parameters $\theta$ to achieve better generalization. Finally, the meta-validation set is used for model selection, while the meta-test set is used for the final evaluation on the selected model.

3.2 Model-Agnostic Meta-Learning

To tackle the problem of fast adaptation to unseen tasks with few examples, we borrow the philosophy and the methodology from MAML [5]. MAML encodes prior knowledge in an initialization and seeks for a “good” common initial set of values for weights of a neural network across tasks. Formally, given a network $f_{\theta}$ with weights $\theta$, MAML learns a set of initial weight values, $\theta$, which will serve as a good starting point for fast adaptation to a new task $\mathcal{T}_{i}$, sampled from a task distribution $p(\mathcal{T})$. Given few examples $\mathcal{D}_{\mathcal{T}_{i}}$ and a loss function $\mathcal{L}_{\mathcal{T}_{i}}$ from the task $\mathcal{T}_{i}$, the network weights are adapted to $\mathcal{T}_{i}$ during inner-loop update as follows:

[TABLE]

To give feedback on the generalization performance of the model with adapted weights $\theta’_{i}$ to each task, the model is evaluated on new examples, $\mathcal{D}^{\mathcal{{}^{\prime}}}_{\mathcal{T}_{i}}$ sampled from the same task $\mathcal{T}_{i}$. The feedback, manifested in the form of loss gradients, is used to update the initialization $\theta$ so that better generalization is achieved:

[TABLE]

3.3 Definition of Conflict

While MAML is elegantly simple, its limitation comes from the very fact that the initialization is shared across a distribution of tasks. Despite the goal of MAML, which is to learn a “good” starting point for fast adaptation to new tasks, the shared initialization, in fact, hinders the fast learning process. This is illustrated by sharp optimization landscape during fast adaptation in Figure 2. This is mainly due to disagreement between tasks on the location of a “good” starting point. We call such disagreement conflict.

At each training iteration, each task $\mathcal{T}_{i}$ takes the initialization closer to the desired location via gradient: $\boldsymbol{u}_{i}=-\nabla_{\theta}\mathcal{L}^{\mathcal{D^{\prime}}}_{\mathcal{T}_{i}}(f_{\theta^{\prime}_{i}})$ during meta-update. However, since MAML shares the initialization, the update is made via gradients accumulated over a batch of tasks $\sum_{i}{\boldsymbol{u_{i}}}$ as in Equation (2). Hence, in the example of two tasks, the conflict occurs between tasks $\mathcal{T}_{i}$ and $\mathcal{T}_{j}$ when their gradient directions, i.e. directions of $\boldsymbol{u}_{i}$ and $\boldsymbol{u}_{j}$, differ. The more their directions differ, the more the initialization update diverges from $\boldsymbol{u}_{i}$ and $\boldsymbol{u}_{j}$, pointing towards the location that is not desirable for both $\mathcal{T}_{i}$ and $\mathcal{T}_{j}$. We refer to this phenomenon as compromise in the initialization.

We define the degree of conflict among tasks to be the average angle between $\boldsymbol{u}_{i}$ and $\sum_{i}{\boldsymbol{u}_{i}}$, which is measured as the average absolute arccosine of the dot product of the normalized vectors, $\mathbb{E}_{\mathcal{T}_{i}\sim p(\mathcal{T})}[\left|cos^{-1}(\boldsymbol{\hat{u}}_{i}\cdot\boldsymbol{v})\right|]$, where $\boldsymbol{\hat{u}}_{i}$ is $\frac{\boldsymbol{u}_{i}}{\left\lVert\boldsymbol{u}_{i}\right\rVert}$ and $\boldsymbol{v}$ is $\frac{\sum_{i}{\boldsymbol{u}_{i}}}{\left\lVert\sum_{i}{\boldsymbol{u}_{i}}\right\rVert}$. Figure 3LABEL:sub@fig:layer_conflict measures the degree of conflict at each epoch and demonstrates that the conflict is indeed more prominent in deeper layers, which aligns with the observation that the deeper layers encode more task-specific features [37].

3.4 Learning to Forget

When the degree of conflict is high, we say the initialization is more compromised, and hence the more difficult it is to learn new tasks quickly, as illustrated by sharp loss landscape in Figure 2. This suggests that the learner finds some part of the initialization to be irrelevant or even detrimental for learning a given task. We thus propose to discard such compromised part of the prior knowledge via attenuating the initialization parameters $\theta$ directly. Then, one may ask which parameter is compromised?

To answer the question, we refer to the previous finding that lower layers of a CNN encode general knowledge while deeper layers contain more task-specific information [37]. Upon this observation, we hypothesize that lower layers do not need much attenuation while deeper layers do. To support our hypothesis, we perform an experiment, shown in Figure 3LABEL:sub@fig:gamma_exp, where we vary the amount of attenuation ($\gamma^{j}$) on each layer to observe how much each layer benefits. As expected, deeper layers favor stronger attenuation while lower layers prefer little to no attenuation. This leads to the second question: How much should the parameters be attenuated layer-wise?

One answer would be to let a model learn to find an optimal set of attenuations. The answers to these two questions lead to our proposal: learn layer-wise attenuation via applying a single learnable parameter $\gamma^{j}$ on the initialization parameters of each layer $\theta^{j}$ as follows:

[TABLE]

where $j$ is the layer index of a neural network. The attenuated initialization $\bar{\theta}$ serves as a new starting point for fast adaptation to tasks. Although this may reduce the extent of compromise that may exist in the original MAML initialization, one may ask if the amount of unnecessary or contradicting information in the initialization is equal across tasks.

Surely, the degree of agreement and disagreement with others differs for different tasks. This can be observed in Figure 3LABEL:sub@fig:task_conflict, where the measured degree of conflict is observed to vary for each task. As a result, there is no consensus between tasks on what the best attenuation is for layer 2, as indicated by different attenuation preferred by each task in Figure 3LABEL:sub@fig:gamma_visual. To resolve such conflict, in addition to the layer-wise attenuation, we propose a task-dependent attenuation. But, this poses another question: What information can be used to make attenuation task-dependent?

We turn to gradients $\nabla_{\theta}\mathcal{L}^{\mathcal{D}_{\mathcal{T}_{i}}}_{\mathcal{T}_{i}}(f_{\theta})$ for the answer. Gradients, used for fast adaptation via gradient descents, not only hold task-specific information but also encode the quality of the initialization with respect to the given task $\mathcal{T}_{i}$ from the perspective of optimization. Thus, we propose to compute gradient $\nabla_{\theta}\mathcal{L}^{\mathcal{D}_{\mathcal{T}_{i}}}_{\mathcal{T}_{i}}(f_{\theta})$ at the initialization and condition a network $g_{\phi}$ on it to generate the task-dependent attenuation:

[TABLE]

where $\gamma_{i}=\{\gamma_{i}^{j}\}$ is the set of layer-wise gammas for the $i$-th task and $g_{\phi}$ is a 3-layer MLP network of parameters $\phi$, with a sigmoid at the end to facilitate attenuation. For the network $g_{\phi}$ to generate layer-wise gammas, the network is conditioned on the layer-wise mean of gradients.

After the initialization is adapted to each task, the network undergoes fast adaptation as in Equation (1) and the initialization is updated as in Equation (2) during training. The overall training procedure is summarized in Algorithm 1.

4 Experiments

In this section, we demonstrate the effectiveness and generalizability of our method through extensive experiments on various problems, including few-shot classification, regression, and reinforcement learning.

4.1 Few-Shot Classification

Two well-known datasets, miniImageNet and tieredImageNet are used for the classification test, both of which are extracted from ImageNet dataset while taking into account for few-learning scenarios. miniImageNet is constructed by randomly selecting 100 classes from the ILSVRC-12 dataset, with each class consisting of 600 images of size 84 $\times$ 84 [35]. The constructed dataset is divided into 3 disjoint subsets: 64 classes for training, 16 for validation, and 20 for test as in [21].

tieredImageNet is a larger subset with 608 classes with 779,165 images of size 84 $\times$ 84 in total. Classes are grouped into 34 categories, according to ImageNet hierarchy. These categories are then split into 3 disjoint sets: 20 categories for training, 6 for validation, and 8 for test. According to [23], this minimizes class similarity between training and test and thus makes the problem more challenging and realistic. Experiments for tieredImageNet and miniImageNet are conducted under typical settings: 5-way 1-shot and 5-way 5-shot classification. For more experiments on other datasets, such as FC100 [20], CIFAR-FS [3], and Meta-Dataset [34], please see the supplementary materials.

4.1.1 Results

The results of our proposed approach, other baselines and existing state-of-the-art approaches on the miniImageNet and tieredImageNet are presented in Table 1 and Table 2, respectively. The proposed method improves MAML by a large margin. We note that our proposed approach remains model-agnostic and achieves better or comparable accuracy to the state-of-the-art approaches with the same backbone, even without fine-tuning. To show generalization of the contribution, we apply L2F to the state-of-the-art MAML-based system LEO and demonstrate the performance improvement, achieving the new state-of-the-art performance.

4.1.2 Ablation Studies

Inner-loop update steps One may argue that the comparisons are not fair because there is one extra adjustment to initialization parameters before inner-loop updates. Table 3 shows ablation studies on the number of inner-loop updates for the proposed and the baseline to demonstrate that the performance gain is not due to an extra number of adjustments to parameters. Rather, the benefits come from forgetting the unnecessary information, helping the learner quickly adapt to new tasks.

Attenuation Scope One may be curious and ask: Is layer-wise attenuation the best way to go? Thus, we analyze different scopes of attenuation; a single attenuation parameter for the whole network, or an individual attenuation parameter for each layer, each filter, and each weight of the network. To focus on investigating which scope of attenuation is most beneficial, we remove the task-dependent part and make the attenuation parameters learnable (with values initialized to be 1), rather than generated by the network $g_{\phi}$.

We perform an ablation study with a 4-layer CNN in 5-way 5-shot classification setting on miniImageNet and present results in Table 4. As expected, the layer-wise attenuation gave the most performance gain. Weight-wise or filter-wise attenuation parameter may have finer control, but these parameters have limited scope in that they do not have information about conflicts that occur at the level of layers or network. On the other hand, layer-wise and network-wise parameters gain information about conflicts in neighbor weights as gradients pass through different weights/filters to reach the same attenuation parameter, since the attenuation parameter is shared by these weights/filters. In the meantime, network-wise parameters do not have enough control and thus perform worse than the layer-wise parameters. In the trade-off between control and information gain, layer-wise has shown to strike the right balance.

Effect of Task-Conditioning Table 4 reports lower performance of layer-wise attenuation model, compared to our full model, MAML+L2F. The only difference between the layer-wise attenuation model and ours is that the layer-wise attenuation model lacks the task-conditioning. One can observe that the most performance gain in our method comes from the attenuation, alluding to the importance of attenuation. Regardless, the task-conditioning does improve the performance as well.

Representation of Task Embedding To verify that gradients contain high-quality information about tasks, we condition the network $g$ on the mean of class prototypes from the pre-trained prototypical network [30](similar to TADAM [20]) as task representation. Table 5 demonstrates that our method with gradients as task representation performs similarly or slightly better than the one with the mean of class prototypes. This exhibits the effectiveness of gradients as task representation from the perspective of the optimization, especially because gradients are simple to obtain and model-agnostic while class prototypes are high-dimensional and not applicable across different domains.

Effect of Attenuation To analyze how much performance gain comes from each part of L2F (i.e. forgetting and task-dependency), we apply each module separately to MAML and present results in Table 6. Since the investigation on effectiveness of task-dependency has already been presented in Table 4, we now focus on the effectiveness of the attenuation, compared to other variant transformations. To that end, we explore different types of task-dependent transformations of the initialization. We start with the simple superset of the attenuation: $\gamma$ without sigmoid (Model 3) such that $\gamma_{i}$ is no longer restricted to be between [math] and $1$, and hence does not facilitate attenuation. We also explore a more flexible option: affine transformation (Model 2), where the network $g_{\phi}$ generates two sets of parameters $\gamma_{i},\delta_{i}$ without sigmoid, which will modulate $f_{\theta}$ via $\gamma^{j}_{i}\theta^{j}+\delta^{j}_{i}$.

Table 6 illustrates that MAML gains performance boost throughout different types of task-dependent transformation, suggesting the benefits of the task-dependency. It is reasonable to expect that more flexibility of transformation (Model 2 and 3) would allow for tasks to bring the initialization to more appropriate location for fast adaptation. Interestingly, the classification accuracy drops as more flexibility is given to the transformation of the initialization. This seeming contradiction underlines the necessity of attenuation (sigmoided $\gamma^{j}_{i}$ in our model), rather than just naïve transformation, of the initialization to forget the compromised part of the prior knowledge encoded in the initialization.

We would like to stress that MAML with task-independent layer- or network-wise attenuation in Table 4 performs better than other task-conditioned transformations in Table 6. This suggests that it is more important to forget the compromised initialization than making it task-adaptive.

4.2 Regression

We investigate the generalizability of the proposed method across domains, starting with evaluating the performance in $k$-shot regression. In $k$-shot regression, the objective is to fit a function, given $k$ samples of points. Following the general settings from [5, 14], the target function is set to be a sinusoid with varying amplitude and phase between tasks. The sampling range of amplitude, frequency, and phase defines a task distribution and is set to be the same for both training and evaluation. Regression is visualized in Figure 4LABEL:sub@fig:regression_fig1, while its prediction, measured in mean-square error (MSE), is presented in Table 7. The results demonstrate that our method not only converges faster but also fits to target functions more accurately.

To further stress the generalization of the MAML+L2F initialization, we extensively increase the degree of conflicts between new tasks and the prior knowledge. To that end, we modify the setting such that amplitude, frequency, and phase are sampled from the non-overlapped ranges for training and evaluation (please refer to the supplementary material for details). In Figure 4LABEL:sub@fig:regression_fig2, our model exhibits higher accuracy and thus claims the better generalization.

4.3 Reinforcement Learning

To further validate the generalizability of L2F, we evaluate the performance in reinforcement learning, specifically in 2D navigation and locomotion environments from [4] as in [5]. We briefly outline the task description below (please refer to the supplementary material for details). Figure 5 presents consistent improvement over MAML across different experiments. This solidifies the generalizability and effectiveness of our proposed method.

4.3.1 2D Navigation

A 2D navigation task is to move an agent from the starting point to the destination point in 2D space, where the reward is defined as the negative of the squared distance to the destination point. We follow the experiment procedure from [5], where they fix the starting point and only vary the location of destination between tasks.

Figure 5LABEL:sub@fig:rl_fig1 presents faster and more precise navigation by our model in both experiment settings, both quantitatively and qualitatively. This solidifies the severity of the conflicts between tasks.

4.3.2 Mujoco

As a more complex reinforcement-learning environment, we experiment on locomotion with the MuJoCo simulator [33], where there are two sets of tasks: a robot is required to move in a particular direction in one set and move with a particular velocity in the other. For both experiments,our method outperforms MAML in large margins as shown in Figure 5LABEL:sub@fig:rl_fig2, LABEL:sub@fig:rl_fig3.

4.4 Loss Landscape

We further validate the effectiveness of our model by illustrating the smoother loss landscape after applying L2F to MAML for the miniImageNet classification tasks, as shown in Figure 2. At the initial stages of training, L2F appears to struggle more, while optimization of MAML seems more stable. This may seem contradictory at first but this actually validates our argument about conflicts between tasks even further. At the beginning, the MAML initialization is not trained enough and thus does not have sufficient prior knowledge of task distribution yet. As training proceeds, the initialization encodes more information about task distribution and encounters conflicts between tasks more frequently. As for L2F, the attenuator $g_{\phi}$ initially does not have enough knowledge about the task distribution and thus generates meaningless attenuation $\gamma_{i}$, deteriorating the initialization. But, the attenuator increasingly encodes more information about the task distribution, generating more appropriate attenuation $\gamma_{i}$ that corresponds to tasks well. The generated $\gamma_{i}$ accordingly allows for a learner to forget the irrelevant part of prior knowledge to help fast adaptation, as illustrated by increasing stability and smoothness of landscape.

5 Conclusion

In this paper, we argue that forcibly sharing a common initialization in MAML induces conflicts across tasks and thus results in the compromised location of the initialization. The severely sharp loss landscape asserts that such compromise makes the MAML initialization a “bad” starting position for fast adaptation. We propose to resolve this discrepancy by facilitating forgetting (attenuating) the irrelevant information that may hinder fast adaptation. Specifically, we propose a task-dependent layer-wise attenuation, named L2F, motivated by the observation that the degree of compromise varies between network layers and tasks. Through extensive experiments across different domains, we validate our argument that selective forgetting greatly facilitates fast adaptation while retaining the simplicity and generalizability of MAML.

A Loss Landscape

In [26], they analyze the stability and smoothness of the optimization landscape by measuring Lipschitzness and the “effective” $\beta$-smoothness of loss. We use these measurements to analyze learning dynamics for both MAML and our proposed method during training on 5-way 5-shot miniImageNet classification tasks. In Figure F, we start with investigating fast-adaptation (or inner-loop) optimization. At each inner-loop update step, we measure variations in loss (Figure FLABEL:sub@sup_fig:loss), the $l_{2}$ difference in gradients (Figure FLABEL:sub@sup_fig:gradient), and the maximum difference in gradient over the distance (Figure FLABEL:sub@sup_fig:beta), as we move to different points along the computed gradient for that gradient descent. We take an average of these values over the number of inner-loop updates and plot them against training iterations. With a similar approach, we also analyze the optimization stability of fast adaptation to validation tasks at every epoch (Figure G). The measurements were averaged over (the number of validation tasks $\times$ the number of inner-loop update steps).

At the initial stages of training, L2F appears to struggle more, while optimization of MAML seems more stable. This may seem contradictory at first but this actually validates our argument about conflicts between tasks even further. At the beginning, the MAML initialization is not trained enough and thus does not have sufficient prior knowledge of task distribution yet. As training proceeds, the initialization encodes more information about task distribution and encounters conflicts between tasks more frequently.

As for L2F, the attenuator network $g_{\phi}$ initially does not have enough knowledge about the task distribution and thus generates meaningless attenuation $\gamma$, deteriorating the initialization. But, the attenuator network increasingly encodes more information about the task distribution, generating more appropriate attenuation $\gamma$ that corresponds to tasks well. The generated $\gamma$ accordingly allows for a learner to forget the irrelevant part of prior knowledge to help fast adaptation, as illustrated by increasing stability and smoothness of landscape in Figure F. The similar observation can be made from G, illustrating the generalizability and the robustness of the proposed method to unseen tasks.

We also investigate the optimization landscape of learning the initialization $\theta$ itself for both MAML and L2F in Figure H. The figure demonstrates that the more stable and smoother landscape is realized by L2F. Because the task-dependent layer-wise attenuation allows for forgetting the irrelevant or conflicting part of prior knowledge present in the initialization $\theta$, it lifts a burden of trying to resolve conflicts between tasks from $\theta$, allowing for more stable training of the initialization itself.

B Extended Experiments on Classification

To further validate that our method consistently provides benefits regardless of scenarios, we compare our method against the baseline on additional datasets that have been recently introduced: FC100 (Fewshot-CIFAR100) [20] and CIFAR-FS (CIFAR100 few-shots) [3]. Both aim for creating more challenging scenarios by using low resolution images ($32\times$32, compared to $84\times 84$ in miniImageNet [21] and tieredImageNet [23]) from CIFAR100 [11]. These two datasets differ in how they create the train/val/test splits of CIFAR100. While CIFAR-FS follows the procedure that was used for miniImageNet, FC100 aligns more with the goal of tieredImageNet in that they try to minimize the amount overlap between splits by splitting based on superclasses. Table H presents results for FC100 and Table I for CIFAR-FS. We also perform additional experiments on Meta-Dataset [34], which is a combination of diverse datasets and hence poses more challenging scenarios, where conflicts can occur among tasks more frequently. Table J shows that our proposed method resolves conflicts better than MAML even under challenging scenarios.

C Regression

C.1 Additional Qualitative results

In Figure I and J, we show a random sample of qualitative results from the $k$-shot sinusoid regression, where $k\in[5,10]$. The target function (or true function) is a sine curve $y(x)=Asin(\omega x+b)$ with the amplitude $A$, frequency $\omega$, phase $b$, and the input range $[-5.0,5.0]$. The sampling range of amplitude, frequency, and phase defines a task distribution. In Figure I, we follow the general settings in [5, 14], where amplitude $A$, frequency $\omega$, and phase $b$ are sampled from the uniform distribution on intervals $[0.1,5.0]$, $[0.8,1.2]$, and $[0,\pi]$, respectively. MAML+L2F demonstrates more accurate regression for both 5 and 10-shot cases, compared to the baseline, MAML. To further stress the generalization of the MAML+L2F initialization, we extensively increase the degree of conflicts between new tasks and the prior knowledge. To that end, we modify the setting such that amplitude, frequency, and phase are sampled from the non-overlapped ranges for training and evaluation. In training, amplitude $A$, frequency $\omega$, and phase $b$ are sampled from the uniform distribution on intervals $[0.1,3.0]$, $[0.8,1.0]$, and $[0,\pi/2]$, respectively. In evaluation, amplitude $A$, frequency $\omega$, and phase $b$ are sampled from the uniform distribution on intervals $[3.0,5.0]$, $[1.0,1.2]$, and $[\pi/2,\pi]$, respectively. In Figure J, our method(MAML+L2F) exhibits better fitting and thus claims the better generalization than MAML for both 5 and 10-shot regression.

C.2 Additional Quantitative results

In Table K, we compare the proposed method against other advanced MAML-based methods, which are generalizable across domains, specifically MuMoMAML and MAML++. As with results on classification, our method consistently outperforms in regression task.

D Reinforcement Learning

D.1 Additional Qualitative results

The qualitative results for the 2D navigation experiments are shown in Figure K. In training, the position of starting point is fixed at $[0,0]$ and the position of destination is randomly sampled from space $[-0.5\times 0.5,-0.5\times 0.5]$, which is the same experiment procedure from [5]. Velocity is clipped to be in the range $[-0.2,0.2]$. In evaluation, we performed experiments with four different task distributions. In Figure K(a), the task distribution for evaluation is the same as for training. On the other hand, as in regression experiment J, we perform additional 3 experiments (Figure K(b), (c), (d) that evaluate models under extreme conditions, where the task distribution for evaluation is chosen to be different from the task distribution for training. In Figure K(b), the staring point is no longer fixed but rather sampled from space $[-0.5\times 0.5,-0.5\times 0.5]$. In Figure K(c), the position of starting point is fixed at $[0,0]$. However, the position of the ending point is sampled from a larger space $[-2.0\times 2.0,-2.0\times 2.0]$. In Figure K(d), both the starting and destination positions are sampled from space $[-2.0\times 2.0,-2.0\times 2.0]$. Overall, our proposed method demonstrates more accurate and robust navigation, compared to the baseline MAML.

E Implementation Details

E.1 classification

E.1.1 Experiment Setup

We use the standard settings [5] for $N$-way $k$-shot classification in both miniImageNet[21] and tieredImageNet[23]. When calculating gradients for fast adaptation to each task, the number of examples $\mathcal{D}$ used is either $N$. The fast adaptation is done via 5 gradient steps with the fixed step size, $\alpha=0.01$ for all models, except LEO and LEO+L2F during both training and evaluation. Gradients for meta-updating the networks $f_{\theta}$ and $g_{\phi}$ are calculated with 15 number of examples $\mathcal{D}^{\prime}$ at each iteration. The MAML and its variants were trained for 50000 iterations in miniImageNet and 125000 in tieredImageNet to account for the larger number of examples as in [15]. The meta batch size of tasks is set to be 2 for 5-shot and 4 for 1-shot, with the exception that the batch size is 1 for ResNet12 in miniImageNet and tieredImageNet. This is due to the limited memory and the heavy computation load from the combination of second-order gradient computation, large image size, and a larger network. As for experiments with LEO, we follow the exact setup from LEO [24]. We only add attenuation process before adaptation in latent space and fine-tuning in parameter space for LEO+L2F.

E.1.2 Network Architecture for $f_{\theta}$

4 conv As with most algorithms [35, 21, 30, 31] that use 4-layer CNN as a backbone, we use 4 layers each of which contains 64-filter 3 $\times$ 3 convolution filters, a batch normalization [8], a Leaky ReLU nonlinearity, and a 2$\times$2 max pooling. Lastly, the classification linear layer and softmax are placed at the end of the network.

ResNet12 As for the ResNet12 architecture, the network consists of 4 residual blocks. each residual block consists of three $3\times 3$ convolution layers. The first two convolution layers are followed by a batch normalization and a ReLU nonlinearity. The last convolution layer is followed by a batch normalization and a skip connection that contains a $1\times 1$ convolution layer and a batch normalization. After a skip connection, a ReLU nonlinearity and a max $2\times 2$ are placed at the end of each residual block.

The number of convolution filters for 4 residual blocks is set to be 64, 128, 256, 512 for 4 residual blocks in the increasing order of depth.

E.1.3 Network Architecture for $g_{\phi}$

$g_{\phi}$ is a 3-layer MLP, with each layer of $l$ hidden units, where $l$ is the number of layers of the main network, $f_{\theta}$. Activation functions are ReLU in between and a sigmoid at the end. The input is layer-wise mean of gradients. As for LEO, the case is a bit different because they perform one adaptation in latent codes and one on decoded classifier weights. Thus, we introduce two 3-layer MLPs, one for each. Again, for each MLP, the numbers of hidden units for each layer is $n$, where $n$ is the dimension of latent codes or the number of classifier weights.

E.2 Regression and RL

The details of the few shot regression and reinforcement learning experiments are listed in Table L.

E.3 System

All experiments were performed on a single NVIDIA GeForce GTX 1080Ti.

F Conflict Within a Task

In this section, we also measure the degree of conflict that exist within task (conflict between examples in the same task), as illustrated in Figure L. As expected, the degree of conflict within task is observed to be lower than the degree of conflict between tasks. This is because the examples within a task are more similar to each other than examples from different tasks. This leads to gradients being more aligned within a task, while gradients between tasks have a larger amount of disagreement.

Acknowledgements

This work was supported by IITP grant funded by the Ministry of Science and ICT of Korea (No. 2017-0-01780), and Hyundai Motor Group through HMG-SNU AI Consortium fund (No. 5264-20190101).