Towards Efficient Capsule Networks

TL;DR

This paper explores combining sparsity with Capsule Networks to reduce computational costs and memory usage, aiming to make them more scalable and efficient for complex tasks.

Contribution

It introduces a pruning approach for Capsule Networks that enhances efficiency and scalability by reducing the number of capsules needed.

Findings

Pruning improves generalization performance.

Reduces memory and computational requirements.

Speeds up training and inference times.

Abstract

From the moment Neural Networks dominated the scene for image processing, the computational complexity needed to solve the targeted tasks skyrocketed: against such an unsustainable trend, many strategies have been developed, ambitiously targeting performance's preservation. Promoting sparse topologies, for example, allows the deployment of deep neural networks models on embedded, resource-constrained devices. Recently, Capsule Networks were introduced to enhance explainability of a model, where each capsule is an explicit representation of an object or its parts. These models show promising results on toy datasets, but their low scalability prevents deployment on more complex tasks. In this work, we explore sparsity besides capsule representations to improve their computational efficiency by reducing the number of capsules. We show how pruning with Capsule Network achieves high generalization with less memory requirements, computational effort, and inference and training time.