PrunedCaps: A Case For Primary Capsules Discrimination

TL;DR

This paper demonstrates that pruning 95% of Primary Capsules in Capsule Networks significantly improves efficiency and speed on multiple datasets without sacrificing accuracy, highlighting dataset-dependent benefits.

Contribution

It introduces a primary capsule pruning method that enhances CapsNet efficiency, reducing computational cost by up to 9.9 times while maintaining accuracy.

Findings

Pruned CapsNets are up to 9.9 times faster.

Pruning removes 95% of capsules with no accuracy loss.

Over 95% reduction in floating-point operations during routing.

Abstract

Capsule Networks (CapsNets) are a generation of image classifiers with proven advantages over Convolutional Neural Networks (CNNs). Better robustness to affine transformation and overlapping image detection are some of the benefits associated with CapsNets. However, CapsNets cannot be classified as resource-efficient deep learning architecture due to the high number of Primary Capsules (PCs). In addition, CapsNets' training and testing are slow and resource hungry. This paper investigates the possibility of Primary Capsules pruning in CapsNets on MNIST handwritten digits, Fashion-MNIST, CIFAR-10, and SVHN datasets. We show that a pruned version of CapsNet performs up to 9.90 times faster than the conventional architecture by removing 95 percent of Capsules without a loss of accuracy. Also, our pruned architecture saves on more than 95.36 percent of floating-point operations in the dynamic routing stage of the architecture. Moreover, we provide insight into why some datasets benefit significantly from pruning while others fall behind.