Graph Routing between Capsules

TL;DR

This paper introduces a graph routing mechanism in capsule networks to model intra-layer relationships, enhancing text classification performance across multiple datasets.

Contribution

It proposes a novel graph routing method that captures intra-layer capsule relationships, improving semantic understanding in text data.

Findings

The approach outperforms existing routing methods in accuracy on five datasets.

Combining bottom-up routing and top-down attention yields the best results.

The method demonstrates strong generalization across different datasets.

Abstract

Routing methods in capsule networks often learn a hierarchical relationship for capsules in successive layers, but the intra-relation between capsules in the same layer is less studied, while this intra-relation is a key factor for the semantic understanding in text data. Therefore, in this paper, we introduce a new capsule network with graph routing to learn both relationships, where capsules in each layer are treated as the nodes of a graph. We investigate strategies to yield adjacency and degree matrix with three different distances from a layer of capsules, and propose the graph routing mechanism between those capsules. We validate our approach on five text classification datasets, and our findings suggest that the approach combining bottom-up routing and top-down attention performs the best. Such an approach demonstrates generalization capability across datasets. Compared to the state-of-the-art routing methods, the improvements in accuracy in the five datasets we used were 0.82, 0.39, 0.07, 1.01, and 0.02, respectively.