Few-shot Learning with LSSVM Base Learner and Transductive Modules

TL;DR

This paper introduces FSLSTM, a few-shot learning method that combines a multi-class LS-SVM base learner with transductive modules, achieving state-of-the-art results by effectively utilizing query sample information.

Contribution

It proposes a novel combination of LS-SVM as a base learner and transductive modules to enhance few-shot learning performance with reduced computational costs.

Findings

Achieves state-of-the-art results on miniImageNet and CIFAR-FS.

Transductive modules significantly improve 1-shot classification accuracy.

LS-SVM base learner balances generalization and computational efficiency.

Abstract

The performance of meta-learning approaches for few-shot learning generally depends on three aspects: features suitable for comparison, the classifier ( base learner ) suitable for low-data scenarios, and valuable information from the samples to classify. In this work, we make improvements for the last two aspects: 1) although there are many effective base learners, there is a trade-off between generalization performance and computational overhead, so we introduce multi-class least squares support vector machine as our base learner which obtains better generation than existing ones with less computational overhead; 2) further, in order to utilize the information from the query samples, we propose two simple and effective transductive modules which modify the support set using the query samples, i.e., adjusting the support samples basing on the attention mechanism and adding the prototypes of the query set with pseudo labels to the support set as the pseudo support samples. These two modules significantly improve the few-shot classification accuracy, especially for the difficult 1-shot setting. Our model, denoted as FSLSTM (Few-Shot learning with LSsvm base learner and Transductive Modules), achieves state-of-the-art performance on miniImageNet and CIFAR-FS few-shot learning benchmarks.