IB-GAN: Disentangled Representation Learning with Information Bottleneck Generative Adversarial Networks

TL;DR

IB-GAN introduces a novel GAN-based model that employs an information bottleneck to achieve disentangled, interpretable representations, outperforming existing methods in disentanglement and sample quality on multiple datasets.

Contribution

The paper presents IB-GAN, a new GAN architecture that incorporates an information bottleneck via an intermediate stochastic layer for improved disentangled representation learning.

Findings

IB-GAN achieves competitive disentanglement scores compared to state-of-the-art methods.

IB-GAN outperforms InfoGAN and beta-VAE in sample quality and diversity.

Experimental results on multiple datasets demonstrate the effectiveness of IB-GAN.

Abstract

We propose a new GAN-based unsupervised model for disentangled representation learning. The new model is discovered in an attempt to utilize the Information Bottleneck (IB) framework to the optimization of GAN, thereby named IB-GAN. The architecture of IB-GAN is partially similar to that of InfoGAN but has a critical difference; an intermediate layer of the generator is leveraged to constrain the mutual information between the input and the generated output. The intermediate stochastic layer can serve as a learnable latent distribution that is trained with the generator jointly in an end-to-end fashion. As a result, the generator of IB-GAN can harness the latent space in a disentangled and interpretable manner. With the experiments on dSprites and Color-dSprites dataset, we demonstrate that IB-GAN achieves competitive disentanglement scores to those of state-of-the-art \b{eta}-VAEs and outperforms InfoGAN. Moreover, the visual quality and the diversity of samples generated by IB-GAN are often better than those by \b{eta}-VAEs and Info-GAN in terms of FID score on CelebA and 3D Chairs dataset.