Training Deep Capsule Networks with Residual Connections

TL;DR

This paper introduces a method to train deeper capsule networks using residual connections, demonstrating improved performance across multiple datasets and routing algorithms, thus enhancing the expressivity and applicability of capsule networks.

Contribution

It proposes a novel approach to enable training of deeper capsule networks with residual connections, addressing depth limitations in existing capsule architectures.

Findings

Deeper capsule networks outperform shallower ones.

Residual connections improve training stability.

Enhanced performance across four datasets.

Abstract

Capsule networks are a type of neural network that have recently gained increased popularity. They consist of groups of neurons, called capsules, which encode properties of objects or object parts. The connections between capsules encrypt part-whole relationships between objects through routing algorithms which route the output of capsules from lower level layers to upper level layers. Capsule networks can reach state-of-the-art results on many challenging computer vision tasks, such as MNIST, Fashion-MNIST, and Small-NORB. However, most capsule network implementations use two to three capsule layers, which limits their applicability as expressivity grows exponentially with depth. One approach to overcome such limitations would be to train deeper network architectures, as it has been done for convolutional neural networks with much increased success. In this paper, we propose a methodology to train deeper capsule networks using residual connections, which is evaluated on four datasets and three different routing algorithms. Our experimental results show that in fact, performance increases when training deeper capsule networks. The source code is available on https://github.com/moejoe95/res-capsnet.