Adaptive Meta-learner via Gradient Similarity for Few-shot Text Classification

TL;DR

This paper introduces AMGS, a novel meta-learning approach that enhances few-shot text classification by reducing overfitting through semantic representation learning and gradient similarity constraints, leading to improved generalization.

Contribution

The paper proposes AMGS, a new adaptive meta-learner that leverages gradient similarity and self-supervised tasks to improve few-shot text classification performance.

Findings

AMGS outperforms existing meta-learning methods on benchmark datasets.

The method effectively reduces overfitting in few-shot scenarios.

Systematic analysis shows the impact of regularization on model performance.

Abstract

Few-shot text classification aims to classify the text under the few-shot scenario. Most of the previous methods adopt optimization-based meta learning to obtain task distribution. However, due to the neglect of matching between the few amount of samples and complicated models, as well as the distinction between useful and useless task features, these methods suffer from the overfitting issue. To address this issue, we propose a novel Adaptive Meta-learner via Gradient Similarity (AMGS) method to improve the model generalization ability to a new task. Specifically, the proposed AMGS alleviates the overfitting based on two aspects: (i) acquiring the potential semantic representation of samples and improving model generalization through the self-supervised auxiliary task in the inner loop, (ii) leveraging the adaptive meta-learner via gradient similarity to add constraints on the gradient obtained by base-learner in the outer loop. Moreover, we make a systematic analysis of the influence of regularization on the entire framework. Experimental results on several benchmarks demonstrate that the proposed AMGS consistently improves few-shot text classification performance compared with the state-of-the-art optimization-based meta-learning approaches.