Space lower bounds for linear prediction in the streaming model

TL;DR

This paper proves that fundamental linear prediction tasks in streaming models require quadratic memory in the dimension, showing inherent limitations for scalable streaming algorithms in these problems.

Contribution

It establishes the first quadratic lower bounds for memory in streaming algorithms solving linear prediction tasks, based on geometric and linear algebraic techniques.

Findings

Linear prediction tasks need quadratic memory in the dimension.

Streaming algorithms cannot efficiently solve these tasks with sub-quadratic memory.

The results are based on lower bounds for orthogonal vector problems and Grassmannian packing estimates.

Abstract

We show that fundamental learning tasks, such as finding an approximate linear separator or linear regression, require memory at least \emph{quadratic} in the dimension, in a natural streaming setting. This implies that such problems cannot be solved (at least in this setting) by scalable memory-efficient streaming algorithms. Our results build on a memory lower bound for a simple linear-algebraic problem -- finding orthogonal vectors -- and utilize the estimates on the packing of the Grassmannian, the manifold of all linear subspaces of fixed dimension.