Generalized Capsule Networks with Trainable Routing Procedure

TL;DR

This paper introduces Generalized Capsule Networks (G-CapsNet) with trainable routing procedures, integrating routing into the training process for improved modeling of spatial features with fewer parameters.

Contribution

The paper proposes a novel G-CapsNet that makes routing coefficients fully trainable, embedding routing into the optimization process, which is a significant advancement over previous fixed routing methods.

Findings

G-CapsNet achieves comparable performance to original CapsNet on MNIST.

No significant difference between capsule packing methods tested.

Stacking multiple capsule layers is explored for future work.

Abstract

CapsNet (Capsule Network) was first proposed by~\citet{capsule} and later another version of CapsNet was proposed by~\citet{emrouting}. CapsNet has been proved effective in modeling spatial features with much fewer parameters. However, the routing procedures in both papers are not well incorporated into the whole training process. The optimal number of routing procedure is misery which has to be found manually. To overcome this disadvantages of current routing procedures in CapsNet, we embed the routing procedure into the optimization procedure with all other parameters in neural networks, namely, make coupling coefficients in the routing procedure become completely trainable. We call it Generalized CapsNet (G-CapsNet). We implement both "full-connected" version of G-CapsNet and "convolutional" version of G-CapsNet. G-CapsNet achieves a similar performance in the dataset MNIST as in the original papers. We also test two capsule packing method (cross feature maps or with feature maps) from previous convolutional layers and see no evident difference. Besides, we also explored possibility of stacking multiple capsule layers. The code is shared on \hyperlink{https://github.com/chenzhenhua986/CAFFE-CapsNet}{CAFFE-CapsNet}.