Semantic-diversity transfer network for generalized zero-shot learning via inner disagreement based OOD detector

TL;DR

This paper introduces SetNet, a novel approach for generalized zero-shot learning that uses multiple local features and an inner disagreement detector to improve knowledge transfer and handle unseen data bias.

Contribution

The paper proposes a Semantic-diversity transfer Network with multiple attention and a projector ensemble, plus an inner disagreement based detector for better GZSL performance.

Findings

SetNet outperforms 30 state-of-the-art methods on three datasets.

The multiple-attention architecture improves semantic consistency.

The ID3M module effectively detects unseen-class data.

Abstract

Zero-shot learning (ZSL) aims to recognize objects from unseen classes, where the kernel problem is to transfer knowledge from seen classes to unseen classes by establishing appropriate mappings between visual and semantic features. The knowledge transfer in many existing works is limited mainly due to the facts that 1) the widely used visual features are global ones but not totally consistent with semantic attributes; 2) only one mapping is learned in existing works, which is not able to effectively model diverse visual-semantic relations; 3) the bias problem in the generalized ZSL (GZSL) could not be effectively handled. In this paper, we propose two techniques to alleviate these limitations. Firstly, we propose a Semantic-diversity transfer Network (SetNet) addressing the first two limitations, where 1) a multiple-attention architecture and a diversity regularizer are proposed to learn multiple local visual features that are more consistent with semantic attributes and 2) a projector ensemble that geometrically takes diverse local features as inputs is proposed to model visual-semantic relations from diverse local perspectives. Secondly, we propose an inner disagreement based domain detection module (ID3M) for GZSL to alleviate the third limitation, which picks out unseen-class data before class-level classification. Due to the absence of unseen-class data in training stage, ID3M employs a novel self-contained training scheme and detects out unseen-class data based on a designed inner disagreement criterion. Experimental results on three public datasets demonstrate that the proposed SetNet with the explored ID3M achieves a significant improvement against $30$ state-of-the-art methods.