`Eyes of a Hawk and Ears of a Fox': Part Prototype Network for Generalized Zero-Shot Learning

TL;DR

This paper introduces a Part Prototype Network (PPN) that leverages region-specific attributes from a pre-trained Vision-Language detector to improve generalized zero-shot learning, especially with localized image proposals.

Contribution

The paper proposes a novel PPN approach that uses region-specific attribute attention for better category recognition in GZSL, differing from traditional global attribute models.

Findings

Achieves promising results on CUB, SUN, and AWA2 datasets.

Outperforms other popular base models in GZSL tasks.

Shows advantages of localized proposals over global attribute attention.

Abstract

Current approaches in Generalized Zero-Shot Learning (GZSL) are built upon base models which consider only a single class attribute vector representation over the entire image. This is an oversimplification of the process of novel category recognition, where different regions of the image may have properties from different seen classes and thus have different predominant attributes. With this in mind, we take a fundamentally different approach: a pre-trained Vision-Language detector (VINVL) sensitive to attribute information is employed to efficiently obtain region features. A learned function maps the region features to region-specific attribute attention used to construct class part prototypes. We conduct experiments on a popular GZSL benchmark consisting of the CUB, SUN, and AWA2 datasets where our proposed Part Prototype Network (PPN) achieves promising results when compared with other popular base models. Corresponding ablation studies and analysis show that our approach is highly practical and has a distinct advantage over global attribute attention when localized proposals are available.