Compressing Deep Neural Networks: A New Hashing Pipeline Using Kac's Random Walk Matrices

TL;DR

This paper introduces a new hashing pipeline using Kac's random walk matrices for compressing deep neural networks, achieving similar accuracy to existing methods while reducing computational complexity.

Contribution

The paper proposes a novel hashing pipeline with Kac's random walk matrices, offering an alternative approach for neural network compression with comparable performance.

Findings

Kac's random walk matrices achieve similar accuracy to existing hashing pipelines.

The new method reduces computational complexity in neural network compression.

The approach maintains angular distance preservation in high-dimensional data.

Abstract

The popularity of deep learning is increasing by the day. However, despite the recent advancements in hardware, deep neural networks remain computationally intensive. Recent work has shown that by preserving the angular distance between vectors, random feature maps are able to reduce dimensionality without introducing bias to the estimator. We test a variety of established hashing pipelines as well as a new approach using Kac's random walk matrices. We demonstrate that this method achieves similar accuracy to existing pipelines.