An Integral Projection-based Semantic Autoencoder for Zero-Shot Learning

TL;DR

This paper introduces an integral projection-based semantic autoencoder (IP-SAE) for zero-shot learning that reduces domain shift issues by preserving visual-semantic relationships through a symmetric transformation, improving classification on benchmark datasets.

Contribution

The paper proposes a novel IP-SAE model that enhances zero-shot learning by integrating a symmetric projection to better preserve visual-semantic data, addressing domain shift and interpretability.

Findings

Outperforms state-of-the-art on four benchmark datasets

Reduces domain shift in zero-shot learning

Provides interpretability of the embedding process

Abstract

Zero-shot Learning (ZSL) classification categorizes or predicts classes (labels) that are not included in the training set (unseen classes). Recent works proposed different semantic autoencoder (SAE) models where the encoder embeds a visual feature vector space into the semantic space and the decoder reconstructs the original visual feature space. The objective is to learn the embedding by leveraging a source data distribution, which can be applied effectively to a different but related target data distribution. Such embedding-based methods are prone to domain shift problems and are vulnerable to biases. We propose an integral projection-based semantic autoencoder (IP-SAE) where an encoder projects a visual feature space concatenated with the semantic space into a latent representation space. We force the decoder to reconstruct the visual-semantic data space. Due to this constraint, the visual-semantic projection function preserves the discriminatory data included inside the original visual feature space. The enriched projection forces a more precise reconstitution of the visual feature space invariant to the domain manifold. Consequently, the learned projection function is less domain-specific and alleviates the domain shift problem. Our proposed IP-SAE model consolidates a symmetric transformation function for embedding and projection, and thus, it provides transparency for interpreting generative applications in ZSL. Therefore, in addition to outperforming state-of-the-art methods considering four benchmark datasets, our analytical approach allows us to investigate distinct characteristics of generative-based methods in the unique context of zero-shot inference.