Time-Space Tradeoffs for Learning from Small Test Spaces: Learning Low Degree Polynomial Functions

TL;DR

This paper extends time-space tradeoff lower bounds to learning problems with small test spaces, specifically for low-degree polynomial functions over finite fields, revealing optimal bounds for computational resources.

Contribution

It introduces a refined matrix norm amplification technique to establish new lower bounds for learning low-degree polynomial functions from random tests.

Findings

Any algorithm learning these polynomials requires either linear space or exponential time.

The bounds match the performance of natural algorithms, indicating their optimality.

New technique improves understanding of resource tradeoffs in learning with limited test spaces.

Abstract

We develop an extension of recently developed methods for obtaining time-space tradeoff lower bounds for problems of learning from random test samples to handle the situation where the space of tests is signficantly smaller than the space of inputs, a class of learning problems that is not handled by prior work. This extension is based on a measure of how matrices amplify the 2-norms of probability distributions that is more refined than the 2-norms of these matrices. As applications that follow from our new technique, we show that any algorithm that learns $m$-variate homogeneous polynomial functions of degree at most $d$ over $\mathbb{F}_2$ from evaluations on randomly chosen inputs either requires space $\Omega(mn)$ or $2^{\Omega(m)}$ time where $n=m^{\Theta(d)}$ is the dimension of the space of such functions. These bounds are asymptotically optimal since they match the tradeoffs achieved by natural learning algorithms for the problems.