Fine-grained Classification via Categorical Memory Networks

TL;DR

This paper introduces a class-specific memory module for fine-grained classification that enhances feature representations by leveraging shared patterns across classes, leading to improved accuracy on multiple benchmarks.

Contribution

The paper proposes a novel categorical memory network that stores class prototypes as moving averages and uses attention to create tailored feature responses for better classification.

Findings

Significant accuracy improvements over baseline CNNs.

Competitive results on four fine-grained classification benchmarks.

Effective use of class prototypes as a discriminative cue.

Abstract

Motivated by the desire to exploit patterns shared across classes, we present a simple yet effective class-specific memory module for fine-grained feature learning. The memory module stores the prototypical feature representation for each category as a moving average. We hypothesize that the combination of similarities with respect to each category is itself a useful discriminative cue. To detect these similarities, we use attention as a querying mechanism. The attention scores with respect to each class prototype are used as weights to combine prototypes via weighted sum, producing a uniquely tailored response feature representation for a given input. The original and response features are combined to produce an augmented feature for classification. We integrate our class-specific memory module into a standard convolutional neural network, yielding a Categorical Memory Network. Our memory module significantly improves accuracy over baseline CNNs, achieving competitive accuracy with state-of-the-art methods on four benchmarks, including CUB-200-2011, Stanford Cars, FGVC Aircraft, and NABirds.