Classical Sequence Match is a Competitive Few-Shot One-Class Learner

TL;DR

This paper demonstrates that classical sequence matching methods, enhanced with meta-learning, can outperform transformer-based models in few-shot one-class classification tasks, offering a less costly alternative.

Contribution

It introduces a classical sequence match approach for few-shot one-class learning and shows its superiority over transformers with meta-learning, reducing training costs.

Findings

Classical sequence match with meta-learning outperforms transformers in few-shot one-class tasks.

Meta-learning increases feature correlation in transformer models, influenced by layers and heads.

Classical methods require less training cost and achieve better performance in the studied setting.

Abstract

Nowadays, transformer-based models gradually become the default choice for artificial intelligence pioneers. The models also show superiority even in the few-shot scenarios. In this paper, we revisit the classical methods and propose a new few-shot alternative. Specifically, we investigate the few-shot one-class problem, which actually takes a known sample as a reference to detect whether an unknown instance belongs to the same class. This problem can be studied from the perspective of sequence match. It is shown that with meta-learning, the classical sequence match method, i.e. Compare-Aggregate, significantly outperforms transformer ones. The classical approach requires much less training cost. Furthermore, we perform an empirical comparison between two kinds of sequence match approaches under simple fine-tuning and meta-learning. Meta-learning causes the transformer models' features to have high-correlation dimensions. The reason is closely related to the number of layers and heads of transformer models. Experimental codes and data are available at https://github.com/hmt2014/FewOne