Capsule Network Performance on Complex Data

TL;DR

This paper evaluates capsule networks' ability to handle complex data, focusing on their rotational invariance and spatial awareness, and compares their performance to CNNs on datasets like CIFAR10.

Contribution

It explores optimal configurations of capsule networks for higher-dimensional data, demonstrating their advantages over traditional CNNs in complex classification tasks.

Findings

Capsule networks achieve lower test error on CIFAR10 compared to CNNs.

Capsule networks demonstrate rotational invariance and spatial awareness.

Optimal configurations improve performance on complex datasets.

Abstract

In recent years, convolutional neural networks (CNN) have played an important role in the field of deep learning. Variants of CNN's have proven to be very successful in classification tasks across different domains. However, there are two big drawbacks to CNN's: their failure to take into account of important spatial hierarchies between features, and their lack of rotational invariance. As long as certain key features of an object are present in the test data, CNN's classify the test data as the object, disregarding features' relative spatial orientation to each other. This causes false positives. The lack of rotational invariance in CNN's would cause the network to incorrectly assign the object another label, causing false negatives. To address this concern, Hinton et al. propose a novel type of neural network using the concept of capsules in a recent paper. With the use of dynamic routing and reconstruction regularization, the capsule network model would be both rotation invariant and spatially aware. The capsule network has shown its potential by achieving a state-of-the-art result of 0.25% test error on MNIST without data augmentation such as rotation and scaling, better than the previous baseline of 0.39%. To further test out the application of capsule networks on data with higher dimensionality, we attempt to find the best set of configurations that yield the optimal test error on CIFAR10 dataset.