Incentivizing Generative Zero-Shot Learning via Outcome-Reward Reinforcement Learning with Visual Cues

TL;DR

This paper introduces RLVC, a reinforcement learning framework with visual cues that improves generative zero-shot learning by producing more task-relevant features, leading to state-of-the-art results.

Contribution

RLVC employs outcome-reward reinforcement learning with visual cues to enhance generative ZSL, addressing task-agnostic feature synthesis issues.

Findings

Achieves 4.7% performance gain on ZSL benchmarks.

Effectively aligns synthesized features with visual prototypes.

Demonstrates state-of-the-art results across three benchmarks.

Abstract

Recent advances in zero-shot learning (ZSL) have demonstrated the potential of generative models. Typically, generative ZSL synthesizes visual features conditioned on semantic prototypes to model the data distribution of unseen classes, followed by training a classifier on the synthesized data. However, the synthesized features often remain task-agnostic, leading to degraded performance. Moreover, inferring a faithful distribution from semantic prototypes alone is insufficient for classes that are semantically similar but visually distinct. To address these and advance ZSL, we propose RLVC, an outcome-reward reinforcement learning RL framework with visual cues for generative ZSL. At its core, RL empowers the generative model to self-evolve, implicitly enhancing its generation capability. In particular, RLVC updates the generative model using an outcome-based reward, encouraging the synthesis of task-relevant features. Furthermore, we introduce class-wise visual cues that (i) align synthesized features with visual prototypes and (ii) stabilize the RL training updates. For the training process, we present a novel cold-start strategy. Comprehensive experiments and analyses on three prevalent ZSL benchmarks demonstrate that RLVC achieves state-of-the-art results with a 4.7% gain.