Convolutional Neural Networks for Text Categorization: Shallow Word-level vs. Deep Character-level

TL;DR

This paper compares shallow word-level CNNs and deep character-level CNNs for text categorization, showing that shallow word-level models achieve better error rates but require more storage, while being faster to compute.

Contribution

It demonstrates that shallow word-level CNNs outperform deep character-level CNNs in error rates on large datasets, providing insights into model efficiency and accuracy.

Findings

Shallow word-level CNNs achieve lower error rates than deep character-level CNNs.

Shallow models require more storage but compute faster.

Results suggest model choice depends on accuracy versus speed/storage trade-offs.

Abstract

This paper reports the performances of shallow word-level convolutional neural networks (CNN), our earlier work (2015), on the eight datasets with relatively large training data that were used for testing the very deep character-level CNN in Conneau et al. (2016). Our findings are as follows. The shallow word-level CNNs achieve better error rates than the error rates reported in Conneau et al., though the results should be interpreted with some consideration due to the unique pre-processing of Conneau et al. The shallow word-level CNN uses more parameters and therefore requires more storage than the deep character-level CNN; however, the shallow word-level CNN computes much faster.