# Training products of expert capsules with mixing by dynamic routing

## TL;DR

This paper introduces an unsupervised learning algorithm for capsule networks that uses an energy-based model with dynamic routing, enabling realistic image generation from learned distributions.

## Contribution

It proposes a novel energy function for capsule networks aligned with dynamic routing, facilitating unsupervised training and image generation.

## Key findings

- Successfully trained capsule networks on standard vision datasets.
- Able to generate realistic images from the learned distribution.
- Demonstrated the effectiveness of energy-based models with dynamic routing.

## Abstract

This study develops an unsupervised learning algorithm for products of expert capsules with dynamic routing. Analogous to binary-valued neurons in Restricted Boltzmann Machines, the magnitude of a squashed capsule firing takes values between zero and one, representing the probability of the capsule being on. This analogy motivates the design of an energy function for capsule networks. In order to have an efficient sampling procedure where hidden layer nodes are not connected, the energy function is made consistent with dynamic routing in the sense of the probability of a capsule firing, and inference on the capsule network is computed with the dynamic routing between capsules procedure. In order to optimize the log-likelihood of the visible layer capsules, the gradient is found in terms of this energy function. The developed unsupervised learning algorithm is used to train a capsule network on standard vision datasets, and is able to generate realistic looking images from its learned distribution.