Learning to Learn with Indispensable Connections

TL;DR

This paper introduces Meta-LTH, a meta-learning approach that identifies essential neural connections using magnitude pruning, leading to improved few-shot learning performance by reducing unnecessary computations.

Contribution

Meta-LTH applies the lottery ticket hypothesis to meta-learning, creating a sub-network with indispensable connections that enhances adaptability and efficiency in few-shot tasks.

Findings

Meta-LTH outperforms first-order MAML on three datasets.

Achieves approximately 2% accuracy improvement on Omniglot.

Reduces over-parameterization by pruning inconsequential connections.

Abstract

Meta-learning aims to solve unseen tasks with few labelled instances. Nevertheless, despite its effectiveness for quick learning in existing optimization-based methods, it has several flaws. Inconsequential connections are frequently seen during meta-training, which results in an over-parameterized neural network. Because of this, meta-testing observes unnecessary computations and extra memory overhead. To overcome such flaws. We propose a novel meta-learning method called Meta-LTH that includes indispensible (necessary) connections. We applied the lottery ticket hypothesis technique known as magnitude pruning to generate these crucial connections that can effectively solve few-shot learning problem. We aim to perform two things: (a) to find a sub-network capable of more adaptive meta-learning and (b) to learn new low-level features of unseen tasks and recombine those features with the already learned features during the meta-test phase. Experimental results show that our proposed Met-LTH method outperformed existing first-order MAML algorithm for three different classification datasets. Our method improves the classification accuracy by approximately 2% (20-way 1-shot task setting) for omniglot dataset.