Momentum Capsule Networks

TL;DR

This paper introduces Momentum Capsule Networks (MoCapsNet), a memory-efficient architecture inspired by reversible residual networks that improves accuracy on standard datasets compared to baseline capsule networks.

Contribution

We propose a novel invertible residual building block framework for capsule networks, enabling reduced memory usage and improved accuracy on multiple datasets.

Findings

MoCapsNet outperforms baseline capsule networks on MNIST, SVHN, CIFAR-10, and CIFAR-100.

MoCapsNet uses significantly less memory than traditional capsule networks.

The framework allows for reversible residual building blocks in capsule architectures.

Abstract

Capsule networks are a class of neural networks that achieved promising results on many computer vision tasks. However, baseline capsule networks have failed to reach state-of-the-art results on more complex datasets due to the high computation and memory requirements. We tackle this problem by proposing a new network architecture, called Momentum Capsule Network (MoCapsNet). MoCapsNets are inspired by Momentum ResNets, a type of network that applies reversible residual building blocks. Reversible networks allow for recalculating activations of the forward pass in the backpropagation algorithm, so those memory requirements can be drastically reduced. In this paper, we provide a framework on how invertible residual building blocks can be applied to capsule networks. We will show that MoCapsNet beats the accuracy of baseline capsule networks on MNIST, SVHN, CIFAR-10 and CIFAR-100 while using considerably less memory. The source code is available on https://github.com/moejoe95/MoCapsNet.