P-CapsNets: a General Form of Convolutional Neural Networks

TL;DR

P-CapsNets are a streamlined, efficient variant of CapsNets that remove routing, replace convolutional layers, and organize capsules into tensors, achieving high accuracy with fewer parameters and improved robustness.

Contribution

This paper introduces P-CapsNets, a new structure that simplifies CapsNets by removing routing and enhancing efficiency, outperforming traditional CapsNets with fewer parameters.

Findings

Achieve over 99% accuracy on MNIST with only 3888 parameters

Outperform CapsNets with varied routing procedures in efficiency and accuracy

Show comparable efficiency to some deep compression models

Abstract

We propose Pure CapsNets (P-CapsNets) which is a generation of normal CNNs structurally. Specifically, we make three modifications to current CapsNets. First, we remove routing procedures from CapsNets based on the observation that the coupling coefficients can be learned implicitly. Second, we replace the convolutional layers in CapsNets to improve efficiency. Third, we package the capsules into rank-3 tensors to further improve efficiency. The experiment shows that P-CapsNets achieve better performance than CapsNets with varied routing procedures by using significantly fewer parameters on MNIST\&CIFAR10. The high efficiency of P-CapsNets is even comparable to some deep compressing models. For example, we achieve more than 99\% percent accuracy on MNIST by using only 3888 parameters. We visualize the capsules as well as the corresponding correlation matrix to show a possible way of initializing CapsNets in the future. We also explore the adversarial robustness of P-CapsNets compared to CNNs.