An Algorithm for Routing Capsules in All Domains
Franz A. Heinsen

TL;DR
This paper introduces a new routing algorithm for capsule networks that improves accuracy and efficiency across vision and language tasks, demonstrating state-of-the-art results and broad applicability.
Contribution
It presents a novel general-purpose routing algorithm for capsule networks, applicable to multiple domains, with demonstrated state-of-the-art performance.
Findings
Achieved 99.1% accuracy on smallNORB with fewer parameters
Attained 58.5% and 95.6% accuracy on Stanford Sentiment Treebank tasks
Provided code and replication instructions for the proposed method
Abstract
Building on recent work on capsule networks, we propose a new, general-purpose form of "routing by agreement" that activates output capsules in a layer as a function of their net benefit to use and net cost to ignore input capsules from earlier layers. To illustrate the usefulness of our routing algorithm, we present two capsule networks that apply it in different domains: vision and language. The first network achieves new state-of-the-art accuracy of 99.1% on the smallNORB visual recognition task with fewer parameters and an order of magnitude less training than previous capsule models, and we find evidence that it learns to perform a form of "reverse graphics." The second network achieves new state-of-the-art accuracies on the root sentences of the Stanford Sentiment Treebank: 58.5% on fine-grained and 95.6% on binary labels with a single-task model that routes frozen embeddings from…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsTopic Modeling · Natural Language Processing Techniques · Multimodal Machine Learning Applications
MethodsLinear Layer · Absolute Position Encodings · Position-Wise Feed-Forward Layer · Residual Connection · Byte Pair Encoding · Dense Connections · Label Smoothing · *Communicated@Fast*How Do I Communicate to Expedia? · Adam · Softmax
