Attribute Distribution Modeling and Semantic-Visual Alignment for Generative Zero-shot Learning

TL;DR

This paper introduces ADiVA, a novel approach for generative zero-shot learning that models attribute distributions and aligns semantic and visual features, significantly improving unseen class recognition.

Contribution

It proposes a joint attribute distribution modeling and semantic-visual alignment framework that enhances feature synthesis for unseen classes in ZSL.

Findings

Achieves 4.7% and 6.1% improvements on AWA2 and SUN datasets.

Outperforms state-of-the-art methods in zero-shot learning.

Can be integrated with existing generative ZSL techniques.

Abstract

Generative zero-shot learning (ZSL) synthesizes features for unseen classes, leveraging semantic conditions to transfer knowledge from seen classes. However, it also introduces two intrinsic challenges: (1) class-level attributes fails to capture instance-specific visual appearances due to substantial intra-class variability, thus causing the class-instance gap; (2) the substantial mismatch between semantic and visual feature distributions, manifested in inter-class correlations, gives rise to the semantic-visual domain gap. To address these challenges, we propose an Attribute Distribution Modeling and Semantic-Visual Alignment (ADiVA) approach, jointly modeling attribute distributions and performing explicit semantic-visual alignment. Specifically, our ADiVA consists of two modules: an Attribute Distribution Modeling (ADM) module that learns a transferable attribute distribution for each class and samples instance-level attributes for unseen classes, and a Visual-Guided Alignment (VGA) module that refines semantic representations to better reflect visual structures. Experiments on three widely used benchmark datasets demonstrate that ADiVA significantly outperforms state-of-the-art methods (e.g., achieving gains of 4.7% and 6.1% on AWA2 and SUN, respectively). Moreover, our approach can serve as a plugin to enhance existing generative ZSL methods.