Improving the Robustness of Capsule Networks to Image Affine Transformations

TL;DR

This paper enhances the robustness of Capsule Networks against affine transformations by proposing affine CapsNets, which significantly improve performance on transformed images without relying on the routing mechanism.

Contribution

The paper introduces affine CapsNets that are more robust to affine transformations and demonstrates their effectiveness without using the routing procedure.

Findings

Affine CapsNets improve accuracy from 79% to 93.21% on AffNIST.

Routing procedure does not significantly contribute to affine robustness.

Affine CapsNets outperform traditional CapsNets on transformed datasets.

Abstract

Convolutional neural networks (CNNs) achieve translational invariance by using pooling operations. However, the operations do not preserve the spatial relationships in the learned representations. Hence, CNNs cannot extrapolate to various geometric transformations of inputs. Recently, Capsule Networks (CapsNets) have been proposed to tackle this problem. In CapsNets, each entity is represented by a vector and routed to high-level entity representations by a dynamic routing algorithm. CapsNets have been shown to be more robust than CNNs to affine transformations of inputs. However, there is still a huge gap between their performance on transformed inputs compared to untransformed versions. In this work, we first revisit the routing procedure by (un)rolling its forward and backward passes. Our investigation reveals that the routing procedure contributes neither to the generalization ability nor to the affine robustness of the CapsNets. Furthermore, we explore the limitations of capsule transformations and propose affine CapsNets (Aff-CapsNets), which are more robust to affine transformations. On our benchmark task, where models are trained on the MNIST dataset and tested on the AffNIST dataset, our Aff-CapsNets improve the benchmark performance by a large margin (from 79% to 93.21%), without using any routing mechanism.