Attentive Graph Neural Networks for Few-Shot Learning

TL;DR

This paper introduces an Attentive Graph Neural Network with a triple-attention mechanism to improve few-shot learning, addressing over-fitting and over-smoothing issues in deep GNNs, and demonstrates superior performance on benchmark datasets.

Contribution

The paper proposes a novel Attentive GNN with triple-attention modules, providing theoretical analysis and extensive experiments showing improved few-shot learning performance.

Findings

Achieves state-of-the-art results on mini-ImageNet and tiered-ImageNet

Effectively mitigates over-fitting and over-smoothing in deep GNNs

Performs well under both inductive and transductive settings

Abstract

Graph Neural Networks (GNN) has demonstrated the superior performance in many challenging applications, including the few-shot learning tasks. Despite its powerful capacity to learn and generalize the model from few samples, GNN usually suffers from severe over-fitting and over-smoothing as the model becomes deep, which limit the scalability. In this work, we propose a novel Attentive GNN to tackle these challenges, by incorporating a triple-attention mechanism, i.e. node self-attention, neighborhood attention, and layer memory attention. We explain why the proposed attentive modules can improve GNN for few-shot learning with theoretical analysis and illustrations. Extensive experiments show that the proposed Attentive GNN model achieves the promising results, comparing to the state-of-the-art GNN- and CNN-based methods for few-shot learning tasks, over the mini-ImageNet and tiered-ImageNet benchmarks, under ConvNet-4 and ResNet-based backbone with both inductive and transductive settings. The codes will be made publicly available.