Neural Networks Compression for Language Modeling

TL;DR

This paper evaluates various compression techniques for RNN-based language models, aiming to reduce size and inference time, especially for mobile applications, using the Penn Treebank dataset.

Contribution

It provides a comparative analysis of pruning, quantization, low-rank factorization, and tensor train decomposition for LSTM networks in language modeling.

Findings

Pruning and quantization significantly reduce model size.

Tensor train decomposition offers a good balance between size and speed.

Low-rank factorization improves inference efficiency.

Abstract

In this paper, we consider several compression techniques for the language modeling problem based on recurrent neural networks (RNNs). It is known that conventional RNNs, e.g, LSTM-based networks in language modeling, are characterized with either high space complexity or substantial inference time. This problem is especially crucial for mobile applications, in which the constant interaction with the remote server is inappropriate. By using the Penn Treebank (PTB) dataset we compare pruning, quantization, low-rank factorization, tensor train decomposition for LSTM networks in terms of model size and suitability for fast inference.