# Limitation of capsule networks

## TL;DR

This paper analyzes the limitations of capsule networks, proving that current routing algorithms restrict their expressivity and negatively impact training, and proposes an improvement to address these issues.

## Contribution

The paper demonstrates that existing routing methods limit capsule networks' expressivity and training, and introduces an incremental improvement to overcome these limitations.

## Key findings

- Routing algorithms prevent capsule networks from distinguishing inputs and negatives.
- Capsule networks are limited to symmetric functions and are not universal approximators.
- The proposed improvement stabilizes training and enhances expressivity.

## Abstract

A recently proposed method in deep learning groups multiple neurons to capsules such that each capsule represents an object or part of an object. Routing algorithms route the output of capsules from lower-level layers to upper-level layers. In this paper, we prove that state-of-the-art routing procedures decrease the expressivity of capsule networks. More precisely, it is shown that EM-routing and routing-by-agreement prevent capsule networks from distinguishing inputs and their negative counterpart. Therefore, only symmetric functions can be expressed by capsule networks, and it can be concluded that they are not universal approximators. We also theoretically motivate and empirically show that this limitation affects the training of deep capsule networks negatively. Therefore, we present an incremental improvement for state-of-the-art routing algorithms that solves the aforementioned limitation and stabilizes the training of capsule networks.