Task-Oriented Channel Attention for Fine-Grained Few-Shot Classification

TL;DR

This paper introduces a task-oriented channel attention method, TDM, with modules SAM, QAM, and IAM, to improve fine-grained few-shot classification by focusing on discriminative features and object-relevant channels.

Contribution

The paper proposes a novel task discrepancy maximization framework with specialized attention modules for fine-grained few-shot classification, enhancing discriminative feature focus and cross-domain performance.

Findings

TDM improves classification accuracy in fine-grained few-shot tasks.

IAM enhances performance in coarse-grained and cross-domain scenarios.

Experimental results validate the effectiveness of the proposed modules.

Abstract

The difficulty of the fine-grained image classification mainly comes from a shared overall appearance across classes. Thus, recognizing discriminative details, such as eyes and beaks for birds, is a key in the task. However, this is particularly challenging when training data is limited. To address this, we propose Task Discrepancy Maximization (TDM), a task-oriented channel attention method tailored for fine-grained few-shot classification with two novel modules Support Attention Module (SAM) and Query Attention Module (QAM). SAM highlights channels encoding class-wise discriminative features, while QAM assigns higher weights to object-relevant channels of the query. Based on these submodules, TDM produces task-adaptive features by focusing on channels encoding class-discriminative details and possessed by the query at the same time, for accurate class-sensitive similarity measure between support and query instances. While TDM influences high-level feature maps by task-adaptive calibration of channel-wise importance, we further introduce Instance Attention Module (IAM) operating in intermediate layers of feature extractors to instance-wisely highlight object-relevant channels, by extending QAM. The merits of TDM and IAM and their complementary benefits are experimentally validated in fine-grained few-shot classification tasks. Moreover, IAM is also shown to be effective in coarse-grained and cross-domain few-shot classifications.