NeuKron: Constant-Size Lossy Compression of Sparse Reorderable Matrices and Tensors

TL;DR

NeuKron is a novel neural network-based method that compresses sparse reorderable matrices and tensors into a constant size, significantly reducing storage while maintaining high approximation accuracy and scalability.

Contribution

It introduces NeuKron, a recurrent neural network approach that generalizes Kronecker products for constant-size lossy compression of sparse matrices and tensors.

Findings

Requires up to five orders of magnitude less space than competitors.

Achieves up to 10x smaller approximation error.

Successfully compresses matrices with over 230 million non-zeros.

Abstract

Many real-world data are naturally represented as a sparse reorderable matrix, whose rows and columns can be arbitrarily ordered (e.g., the adjacency matrix of a bipartite graph). Storing a sparse matrix in conventional ways requires an amount of space linear in the number of non-zeros, and lossy compression of sparse matrices (e.g., Truncated SVD) typically requires an amount of space linear in the number of rows and columns. In this work, we propose NeuKron for compressing a sparse reorderable matrix into a constant-size space. NeuKron generalizes Kronecker products using a recurrent neural network with a constant number of parameters. NeuKron updates the parameters so that a given matrix is approximated by the product and reorders the rows and columns of the matrix to facilitate the approximation. The updates take time linear in the number of non-zeros in the input matrix, and the approximation of each entry can be retrieved in logarithmic time. We also extend NeuKron to compress sparse reorderable tensors (e.g. multi-layer graphs), which generalize matrices. Through experiments on ten real-world datasets, we show that NeuKron is (a) Compact: requiring up to five orders of magnitude less space than its best competitor with similar approximation errors, (b) Accurate: giving up to 10x smaller approximation error than its best competitors with similar size outputs, and (c) Scalable: successfully compressing a matrix with over 230 million non-zero entries.