Multi-level Second-order Few-shot Learning

TL;DR

This paper introduces MlSo, a multi-level, multi-scale second-order few-shot learning network that leverages feature reweighting and self-supervision to improve classification and recognition across diverse datasets.

Contribution

The paper presents a novel multi-level, multi-scale second-order pooling framework with feature matching and self-supervised discrimination for few-shot learning.

Findings

Achieves competitive results on standard image classification datasets.

Demonstrates effectiveness on fine-grained and action recognition datasets.

End-to-end training with a simple architecture.

Abstract

We propose a Multi-level Second-order (MlSo) few-shot learning network for supervised or unsupervised few-shot image classification and few-shot action recognition. We leverage so-called power-normalized second-order base learner streams combined with features that express multiple levels of visual abstraction, and we use self-supervised discriminating mechanisms. As Second-order Pooling (SoP) is popular in image recognition, we employ its basic element-wise variant in our pipeline. The goal of multi-level feature design is to extract feature representations at different layer-wise levels of CNN, realizing several levels of visual abstraction to achieve robust few-shot learning. As SoP can handle convolutional feature maps of varying spatial sizes, we also introduce image inputs at multiple spatial scales into MlSo. To exploit the discriminative information from multi-level and multi-scale features, we develop a Feature Matching (FM) module that reweights their respective branches. We also introduce a self-supervised step, which is a discriminator of the spatial level and the scale of abstraction. Our pipeline is trained in an end-to-end manner. With a simple architecture, we demonstrate respectable results on standard datasets such as Omniglot, mini-ImageNet, tiered-ImageNet, Open MIC, fine-grained datasets such as CUB Birds, Stanford Dogs and Cars, and action recognition datasets such as HMDB51, UCF101, and mini-MIT.