Attentional Meta-learners for Few-shot Polythetic Classification

TL;DR

This paper introduces attentional meta-learners with a self-attention feature-selection mechanism that efficiently handles polythetic classification tasks, especially in the presence of irrelevant features, improving few-shot learning performance.

Contribution

It proposes a novel self-attention feature-selection method for attentional meta-learners, enabling better handling of task-irrelevant features in few-shot classification.

Findings

Attentional classifiers are inherently polythetic and require linear embedding dimensions.

The proposed feature-selection mechanism improves robustness against irrelevant features.

The approach outperforms existing methods on Boolean, synthetic, and real-world few-shot tasks.

Abstract

Polythetic classifications, based on shared patterns of features that need neither be universal nor constant among members of a class, are common in the natural world and greatly outnumber monothetic classifications over a set of features. We show that threshold meta-learners, such as Prototypical Networks, require an embedding dimension that is exponential in the number of task-relevant features to emulate these functions. In contrast, attentional classifiers, such as Matching Networks, are polythetic by default and able to solve these problems with a linear embedding dimension. However, we find that in the presence of task-irrelevant features, inherent to meta-learning problems, attentional models are susceptible to misclassification. To address this challenge, we propose a self-attention feature-selection mechanism that adaptively dilutes non-discriminative features. We demonstrate the effectiveness of our approach in meta-learning Boolean functions, and synthetic and real-world few-shot learning tasks.