$\beta$-CapsNet: Learning Disentangled Representation for CapsNet by Information Bottleneck

TL;DR

This paper introduces $eta$-CapsNet, a novel framework that employs an information bottleneck constraint to learn disentangled, interpretable capsule representations, achieving state-of-the-art results in both supervised and unsupervised settings.

Contribution

The paper proposes a new $eta$-CapsNet model with a variational inference approach for disentanglement and a dynamic routing algorithm for unsupervised learning, advancing capsule network interpretability.

Findings

Achieves state-of-the-art disentanglement performance on complex datasets.

Demonstrates effective trade-off between disentanglement, reconstruction, and classification.

Unsupervised $eta$-CapsNet outperforms baselines in disentanglement tasks.

Abstract

We present a framework for learning disentangled representation of CapsNet by information bottleneck constraint that distills information into a compact form and motivates to learn an interpretable factorized capsule. In our $\beta$-CapsNet framework, hyperparameter $\beta$ is utilized to trade-off disentanglement and other tasks, variational inference is utilized to convert the information bottleneck term into a KL divergence that is approximated as a constraint on the mean of the capsule. For supervised learning, class independent mask vector is used for understanding the types of variations synthetically irrespective of the image class, we carry out extensive quantitative and qualitative experiments by tuning the parameter $\beta$ to figure out the relationship between disentanglement, reconstruction and classfication performance. Furthermore, the unsupervised $\beta$-CapsNet and the corresponding dynamic routing algorithm is proposed for learning disentangled capsule in an unsupervised manner, extensive empirical evaluations suggest that our $\beta$-CapsNet achieves state-of-the-art disentanglement performance compared to CapsNet and various baselines on several complex datasets both in supervision and unsupervised scenes.