Focus-Consistent Multi-Level Aggregation for Compositional Zero-Shot Learning

TL;DR

This paper introduces FOMA, a novel multi-level aggregation method for compositional zero-shot learning that enhances feature diversity and focus consistency across branches, leading to improved recognition of unseen attribute-object pairs.

Contribution

The paper proposes FOMA, which uses multi-level feature aggregation and focus-consistent constraints to better model relationships among branches in CZSL, outperforming existing methods.

Findings

FOMA achieves state-of-the-art results on three benchmark datasets.

The method improves class distinction by focusing on informative regions.

Extensive experiments validate the effectiveness of the proposed approach.

Abstract

To transfer knowledge from seen attribute-object compositions to recognize unseen ones, recent compositional zero-shot learning (CZSL) methods mainly discuss the optimal classification branches to identify the elements, leading to the popularity of employing a three-branch architecture. However, these methods mix up the underlying relationship among the branches, in the aspect of consistency and diversity. Specifically, consistently providing the highest-level features for all three branches increases the difficulty in distinguishing classes that are superficially similar. Furthermore, a single branch may focus on suboptimal regions when spatial messages are not shared between the personalized branches. Recognizing these issues and endeavoring to address them, we propose a novel method called Focus-Consistent Multi-Level Aggregation (FOMA). Our method incorporates a Multi-Level Feature Aggregation (MFA) module to generate personalized features for each branch based on the image content. Additionally, a Focus-Consistent Constraint encourages a consistent focus on the informative regions, thereby implicitly exchanging spatial information between all branches. Extensive experiments on three benchmark datasets (UT-Zappos, C-GQA, and Clothing16K) demonstrate that our FOMA outperforms SOTA.