Domain-aware Visual Bias Eliminating for Generalized Zero-Shot Learning

TL;DR

This paper introduces a novel domain-aware visual bias elimination network for generalized zero-shot learning, which constructs dual visual representations to improve unseen class recognition and reduce domain bias.

Contribution

The proposed DVBE network uniquely combines semantic-free and semantic-aligned visual representations with adaptive mechanisms for better domain separation and unseen class prediction.

Findings

Outperforms existing methods by an average of 5.7% on five benchmarks.

Effectively filters unseen images through semantic-free representation.

Reduces biased recognition towards seen classes significantly.

Abstract

Recent methods focus on learning a unified semantic-aligned visual representation to transfer knowledge between two domains, while ignoring the effect of semantic-free visual representation in alleviating the biased recognition problem. In this paper, we propose a novel Domain-aware Visual Bias Eliminating (DVBE) network that constructs two complementary visual representations, i.e., semantic-free and semantic-aligned, to treat seen and unseen domains separately. Specifically, we explore cross-attentive second-order visual statistics to compact the semantic-free representation, and design an adaptive margin Softmax to maximize inter-class divergences. Thus, the semantic-free representation becomes discriminative enough to not only predict seen class accurately but also filter out unseen images, i.e., domain detection, based on the predicted class entropy. For unseen images, we automatically search an optimal semantic-visual alignment architecture, rather than manual designs, to predict unseen classes. With accurate domain detection, the biased recognition problem towards the seen domain is significantly reduced. Experiments on five benchmarks for classification and segmentation show that DVBE outperforms existing methods by averaged 5.7% improvement.