Variational Capsule Encoder

TL;DR

This paper introduces Bayesian capsules, a novel variational encoder architecture using capsule networks, which improves feature representation, image reconstruction, and classification performance over traditional VAEs on MNIST datasets.

Contribution

It presents the first integration of capsule networks with variational auto-encoders, demonstrating enhanced latent space representation and performance.

Findings

Improved image reconstruction and classification on MNIST and Fashion-MNIST.

Better latent space separation of classes compared to traditional VAEs.

Enhanced representation learning with increased latent space dimensions.

Abstract

We propose a novel capsule network based variational encoder architecture, called Bayesian capsules (B-Caps), to modulate the mean and standard deviation of the sampling distribution in the latent space. We hypothesized that this approach can learn a better representation of features in the latent space than traditional approaches. Our hypothesis was tested by using the learned latent variables for image reconstruction task, where for MNIST and Fashion-MNIST datasets, different classes were separated successfully in the latent space using our proposed model. Our experimental results have shown improved reconstruction and classification performances for both datasets adding credence to our hypothesis. We also showed that by increasing the latent space dimension, the proposed B-Caps was able to learn a better representation when compared to the traditional variational auto-encoders (VAE). Hence our results indicate the strength of capsule networks in representation learning which has never been examined under the VAE settings before.