Dynamic Visual-semantic Alignment for Zero-shot Learning with Ambiguous Labels

TL;DR

This paper introduces DVSA, a robust zero-shot learning framework that effectively handles ambiguous labels through dynamic alignment, contrastive optimization, and label disambiguation, improving recognition of unseen classes.

Contribution

The paper proposes a novel dynamic visual-semantic alignment method with label disambiguation for zero-shot learning under ambiguous supervision, enhancing robustness and accuracy.

Findings

DVSA outperforms existing methods on standard benchmarks.

The dynamic label disambiguation reduces noise and improves generalization.

Contrastive MI optimization enhances attribute discriminability.

Abstract

Zero-shot learning (ZSL) aims to recognize unseen classes without visual instances. However, existing methods usually assume clean labels, overlooking real-world label noise and ambiguity, which degrades performance. To bridge this gap, we propose the Dynamic Visual-semantic Alignment (DVSA), a robust ZSL framework for learning from ambiguous labels. DVSA uses a bidirectional visual-semantic alignment module with attention to mutually calibrate visual features and attribute prototypes, and a contrastive optimization grounded in Mutual Information (MI) at the attribute level to strengthen discriminative, semantically consistent attributes. In addition, a dynamic label disambiguation mechanism iteratively corrects noisy supervision while preserving semantic consistency, narrowing the instance-label gap, and improving generalization. Extensive experiments on standard benchmarks verify that DVSA achieves stronger performance under ambiguous supervision.