Hierarchically Structured Meta-learning

TL;DR

This paper introduces a hierarchical meta-learning approach that clusters tasks to better customize knowledge, improving learning efficiency and performance in few-shot and continual learning scenarios.

Contribution

It proposes a novel hierarchical task clustering method for meta-learning, enhancing knowledge transfer and adaptation across diverse task groups.

Findings

Achieves state-of-the-art results in toy regression tasks.

Outperforms existing methods in few-shot image classification.

Effectively handles task heterogeneity and continual learning environments.

Abstract

In order to learn quickly with few samples, meta-learning utilizes prior knowledge learned from previous tasks. However, a critical challenge in meta-learning is task uncertainty and heterogeneity, which can not be handled via globally sharing knowledge among tasks. In this paper, based on gradient-based meta-learning, we propose a hierarchically structured meta-learning (HSML) algorithm that explicitly tailors the transferable knowledge to different clusters of tasks. Inspired by the way human beings organize knowledge, we resort to a hierarchical task clustering structure to cluster tasks. As a result, the proposed approach not only addresses the challenge via the knowledge customization to different clusters of tasks, but also preserves knowledge generalization among a cluster of similar tasks. To tackle the changing of task relationship, in addition, we extend the hierarchical structure to a continual learning environment. The experimental results show that our approach can achieve state-of-the-art performance in both toy-regression and few-shot image classification problems.