Fine-Grained Zero-Shot Learning with Attribute-Centric Representations

TL;DR

This paper introduces Attribute-Centric Representations (ACR) for fine-grained zero-shot learning, effectively disentangling attributes to improve recognition of unseen categories, and achieves state-of-the-art results on benchmark datasets.

Contribution

The paper proposes a novel framework with mixture-of-experts components to learn attribute-disentangled representations for zero-shot learning.

Findings

Achieves state-of-the-art results on CUB, AwA2, and SUN datasets.

Effectively disentangles visual attributes for better zero-shot classification.

Demonstrates the effectiveness of attribute-centric representations in fine-grained recognition.

Abstract

Recognizing unseen fine-grained categories demands a model that can distinguish subtle visual differences. This is typically achieved by transferring visual-attribute relationships from seen classes to unseen classes. The core challenge is attribute entanglement, where conventional models collapse distinct attributes like color, shape, and texture into a single visual embedding. This causes interference that masks these critical distinctions. The post-hoc solutions of previous work are insufficient, as they operate on representations that are already mixed. We propose a zero-shot learning framework that learns AttributeCentric Representations (ACR) to tackle this problem by imposing attribute disentanglement during representation learning. ACR is achieved with two mixture-of-experts components, including Mixture of Patch Experts (MoPE) and Mixture of Attribute Experts (MoAE). First, MoPE is inserted into the transformer using a dual-level routing mechanism to conditionally dispatch image patches to specialized experts. This ensures coherent attribute families are processed by dedicated experts. Finally, the MoAE head projects these expert-refined features into sparse, partaware attribute maps for robust zero-shot classification. On zero-shot learning benchmark datasets CUB, AwA2, and SUN, our ACR achieves consistent state-of-the-art results.