Disentangled Generation with Information Bottleneck for Few-Shot Learning

TL;DR

DisGenIB introduces an information bottleneck framework for disentangled sample generation in few-shot learning, improving sample quality by enhancing discrimination and diversity, and effectively utilizing priors.

Contribution

The paper proposes a novel IB-based framework for disentangled generation in FSL, demonstrating its generality and superiority over existing methods.

Findings

Outperforms existing FSL generative methods on benchmarks

Effectively utilizes priors for better disentanglement

Theoretically unifies previous generative and disentanglement methods

Abstract

Few-shot learning (FSL), which aims to classify unseen classes with few samples, is challenging due to data scarcity. Although various generative methods have been explored for FSL, the entangled generation process of these methods exacerbates the distribution shift in FSL, thus greatly limiting the quality of generated samples. To these challenges, we propose a novel Information Bottleneck (IB) based Disentangled Generation Framework for FSL, termed as DisGenIB, that can simultaneously guarantee the discrimination and diversity of generated samples. Specifically, we formulate a novel framework with information bottleneck that applies for both disentangled representation learning and sample generation. Different from existing IB-based methods that can hardly exploit priors, we demonstrate our DisGenIB can effectively utilize priors to further facilitate disentanglement. We further prove in theory that some previous generative and disentanglement methods are special cases of our DisGenIB, which demonstrates the generality of the proposed DisGenIB. Extensive experiments on challenging FSL benchmarks confirm the effectiveness and superiority of DisGenIB, together with the validity of our theoretical analyses. Our codes will be open-source upon acceptance.