Quantum-inspired algorithm applied to extreme learning

TL;DR

This paper applies quantum-inspired singular value decomposition to extreme learning, demonstrating significant speedups and analyzing the importance of matrix non-uniformity for sampling strategies in machine learning tasks.

Contribution

It introduces the application of quantum-inspired SVD to extreme learning, showing practical speed advantages and insights into sampling strategies based on matrix properties.

Findings

Quantum-inspired SVD is significantly faster than exact SVD for high-dimensional features.

Uniform sampling suffices for random neural network features due to matrix uniformity.

Non-uniformity in matrix elements enhances the effectiveness of norm-based sampling.

Abstract

Quantum-inspired singular value decomposition (SVD) is a technique to perform SVD in logarithmic time with respect to the dimension of a matrix, given access to the matrix embedded in a segment-tree data structure. The speedup is possible through the efficient sampling of matrix elements according to their norms. Here, we apply it to extreme learning which is a machine learning framework that performs linear regression using random feature vectors generated through a random neural network. The extreme learning is suited for the application of quantum-inspired SVD in that it first requires transforming each data to a random feature during which we can construct the data structure with a logarithmic overhead with respect to the number of data. We implement the algorithm and observe that it works order-of-magnitude faster than the exact SVD when we use high-dimensional feature vectors. However, we also observe that, for random features generated by random neural networks, we can replace the norm-based sampling in the quantum-inspired algorithm with uniform sampling to obtain the same level of test accuracy due to the uniformity of the matrix in this case. The norm-based sampling becomes effective for more non-uniform matrices obtained by optimizing the feature mapping. It implies the non-uniformity of matrix elements is a key property of the quantum-inspired SVD. This work is a first step toward the practical application of the quantum-inspired algorithm.