Compositional Coding Capsule Network with K-Means Routing for Text Classification

TL;DR

This paper introduces a novel neural network for text classification that combines compositional weighted coding and a K-Means based routing algorithm within a capsule network, achieving high accuracy with fewer parameters.

Contribution

It proposes a new compositional weighted coding method and a K-Means clustering-based routing algorithm for capsule networks applied to text classification.

Findings

Achieves competitive accuracy on eight datasets

Uses significantly fewer parameters than state-of-the-art methods

Demonstrates effective modeling of word embedding relationships

Abstract

Text classification is a challenging problem which aims to identify the category of texts. In the process of training, word embeddings occupy a large part of parameters. Under the limitation of limited computing resources, it indirectly limits the ability of subsequent network designs. In order to reduce the number of parameters, the compositional coding mechanism has been proposed recently. Based on this, this paper further explores compositional coding and proposes a compositional weighted coding method. And we apply capsule network to model the relationship between word embeddings, a new routing algorithm, which is based on k-means clustering theory, is proposed to fully mine the relationship between word embeddings. Combined with our compositional weighted coding method and the routing algorithm, we design a neural network for text classification. Experiments conducted on eight challenging text classification datasets show that the proposed method achieves competitive accuracy compared to the state-of-the-art approach with significantly fewer parameters.