EEML: Ensemble Embedded Meta-learning

TL;DR

EEML introduces an ensemble meta-learning approach that organizes prior knowledge into specialized experts, improving few-shot learning by handling task heterogeneity through task embedding and expert collaboration.

Contribution

The paper proposes a novel ensemble embedded meta-learning algorithm that explicitly manages diverse tasks with specialized experts and a task embedding mechanism.

Findings

Outperforms recent state-of-the-art methods in few-shot learning.

Effectively handles task heterogeneity through expert specialization.

Demonstrates the importance of differentiation and cooperation among experts.

Abstract

To accelerate learning process with few samples, meta-learning resorts to prior knowledge from previous tasks. However, the inconsistent task distribution and heterogeneity is hard to be handled through a global sharing model initialization. In this paper, based on gradient-based meta-learning, we propose an ensemble embedded meta-learning algorithm (EEML) that explicitly utilizes multi-model-ensemble to organize prior knowledge into diverse specific experts. We rely on a task embedding cluster mechanism to deliver diverse tasks to matching experts in training process and instruct how experts collaborate in test phase. As a result, the multi experts can focus on their own area of expertise and cooperate in upcoming task to solve the task heterogeneity. The experimental results show that the proposed method outperforms recent state-of-the-arts easily in few-shot learning problem, which validates the importance of differentiation and cooperation.