Efficient Tensor Contraction via Fast Count Sketch

TL;DR

This paper introduces a fast count sketch (FCS) method that improves tensor sketching accuracy and speed by using multiple shorter hash functions and FFT, outperforming existing methods in tensor applications.

Contribution

The paper proposes FCS, a novel tensor sketching technique that enhances accuracy and efficiency over existing methods like CS, TS, and HCS, especially for tensors with CPD.

Findings

FCS achieves higher approximation accuracy than TS and HCS.

FCS significantly reduces computational time for tensor operations.

Experimental results confirm FCS's superior performance in tensor decomposition and compression tasks.

Abstract

Sketching uses randomized Hash functions for dimensionality reduction and acceleration. The existing sketching methods, such as count sketch (CS), tensor sketch (TS), and higher-order count sketch (HCS), either suffer from low accuracy or slow speed in some tensor based applications. In this paper, the proposed fast count sketch (FCS) applies multiple shorter Hash functions based CS to the vector form of the input tensor, which is more accurate than TS since the spatial information of the input tensor can be preserved more sufficiently. When the input tensor admits CANDECOMP/PARAFAC decomposition (CPD), FCS can accelerate CS and HCS by using fast Fourier transform, which exhibits a computational complexity asymptotically identical to TS for low-order tensors. The effectiveness of FCS is validated by CPD, tensor regression network compression, and Kronecker product compression. Experimental results show its superior performance in terms of approximation accuracy and computational efficiency.