Towards optimal sampling for learning sparse approximation in high dimensions

TL;DR

This paper investigates how sampling strategies influence the efficiency of learning sparse high-dimensional functions, proposing near-optimal methods for known and unknown sparse representations with practical examples.

Contribution

It introduces a near-complete sampling approach for known sparse representations and a general construction for unknown cases, improving over standard Monte Carlo methods.

Findings

Sampling strategies significantly affect sample complexity.

Designed sampling measures outperform standard Monte Carlo.

New approximation procedure effective on irregular domains.

Abstract

In this chapter, we discuss recent work on learning sparse approximations to high-dimensional functions on data, where the target functions may be scalar-, vector- or even Hilbert space-valued. Our main objective is to study how the sampling strategy affects the sample complexity -- that is, the number of samples that suffice for accurate and stable recovery -- and to use this insight to obtain optimal or near-optimal sampling procedures. We consider two settings. First, when a target sparse representation is known, in which case we present a near-complete answer based on drawing independent random samples from carefully-designed probability measures. Second, we consider the more challenging scenario when such representation is unknown. In this case, while not giving a full answer, we describe a general construction of sampling measures that improves over standard Monte Carlo sampling. We present examples using algebraic and trigonometric polynomials, and for the former, we also introduce a new procedure for function approximation on irregular (i.e., nontensorial) domains. The effectiveness of this procedure is shown through numerical examples. Finally, we discuss a number of structured sparsity models, and how they may lead to better approximations.