Approximating Subadditive Hadamard Functions on Implicit Matrices

TL;DR

This paper develops new techniques for approximating entrywise functions on implicit matrices in streaming models, improving space complexity for tasks like independence testing between vectors.

Contribution

It introduces a general framework for approximating Hadamard functions on implicit matrices, leading to improved space bounds for independence testing in streaming data.

Findings

Achieved a space complexity of O(log^{12}(n) log^{2}(nm/ε) ε^{-7}) for independence testing.

Extended the approximation techniques to a broader class of implicit matrices beyond outer product form.

Provided theoretical guarantees for the accuracy of the proposed approximation methods.

Abstract

An important challenge in the streaming model is to maintain small-space approximations of entrywise functions performed on a matrix that is generated by the outer product of two vectors given as a stream. In other works, streams typically define matrices in a standard way via a sequence of updates, as in the work of Woodruff (2014) and others. We describe the matrix formed by the outer product, and other matrices that do not fall into this category, as implicit matrices. As such, we consider the general problem of computing over such implicit matrices with Hadamard functions, which are functions applied entrywise on a matrix. In this paper, we apply this generalization to provide new techniques for identifying independence between two vectors in the streaming model. The previous state of the art algorithm of Braverman and Ostrovsky (2010) gave a $(1 \pm \epsilon)$-approximation for the $L_1$ distance between the product and joint distributions, using space $O(\log^{1024}(nm) \epsilon^{-1024})$, where $m$ is the length of the stream and $n$ denotes the size of the universe from which stream elements are drawn. Our general techniques include the $L_1$ distance as a special case, and we give an improved space bound of $O(\log^{12}(n) \log^{2}({nm \over \epsilon})\epsilon^{-7})$.