Multi-Granularity Mutual Refinement Network for Zero-Shot Learning

TL;DR

This paper introduces Mg-MRN, a novel network that refines visual features at multiple granularities and models their interactions to improve zero-shot learning accuracy.

Contribution

The paper proposes a multi-granularity feature extraction and cross-granularity interaction framework for enhanced zero-shot learning performance.

Findings

Outperforms existing ZSL methods on benchmark datasets.

Effectively captures multi-level visual features and their interactions.

Improves discriminability and transferability of visual features.

Abstract

Zero-shot learning (ZSL) aims to recognize unseen classes with zero samples by transferring semantic knowledge from seen classes. Current approaches typically correlate global visual features with semantic information (i.e., attributes) or align local visual region features with corresponding attributes to enhance visual-semantic interactions. Although effective, these methods often overlook the intrinsic interactions between local region features, which can further improve the acquisition of transferable and explicit visual features. In this paper, we propose a network named Multi-Granularity Mutual Refinement Network (Mg-MRN), which refine discriminative and transferable visual features by learning decoupled multi-granularity features and cross-granularity feature interactions. Specifically, we design a multi-granularity feature extraction module to learn region-level discriminative features through decoupled region feature mining. Then, a cross-granularity feature fusion module strengthens the inherent interactions between region features of varying granularities. This module enhances the discriminability of representations at each granularity level by integrating region representations from adjacent hierarchies, further improving ZSL recognition performance. Extensive experiments on three popular ZSL benchmark datasets demonstrate the superiority and competitiveness of our proposed Mg-MRN method. Our code is available at https://github.com/NingWang2049/Mg-MRN.