byteSteady: Fast Classification Using Byte-Level n-Gram Embeddings

TL;DR

byteSteady is a fast, byte-level n-gram embedding-based classification model effective for text and DNA data, offering competitive results and a novel trade-off between compression and accuracy.

Contribution

Introduces byteSteady, a new fast classification method using byte-level n-gram embeddings and hashing, applicable to language and non-language data.

Findings

Achieves competitive results on text and DNA classification tasks.

Simple Huffman coding compression does not significantly reduce accuracy.

Demonstrates applicability to both language and non-language data.

Abstract

This article introduces byteSteady -- a fast model for classification using byte-level n-gram embeddings. byteSteady assumes that each input comes as a sequence of bytes. A representation vector is produced using the averaged embedding vectors of byte-level n-grams, with a pre-defined set of n. The hashing trick is used to reduce the number of embedding vectors. This input representation vector is then fed into a linear classifier. A straightforward application of byteSteady is text classification. We also apply byteSteady to one type of non-language data -- DNA sequences for gene classification. For both problems we achieved competitive classification results against strong baselines, suggesting that byteSteady can be applied to both language and non-language data. Furthermore, we find that simple compression using Huffman coding does not significantly impact the results, which offers an accuracy-speed trade-off previously unexplored in machine learning.