When compressive learning fails: blame the decoder or the sketch?

TL;DR

This paper investigates the challenges of compressive learning, focusing on the non-convex optimization landscape and the effectiveness of heuristics like CLOMPR through numerical simulations.

Contribution

It provides an analysis of why compressive learning can fail, highlighting issues with the decoder and sketch, and evaluates heuristics in this context.

Findings

Non-convex landscape affects heuristic performance

CLOMPR's limitations are linked to optimization challenges

Numerical simulations reveal conditions for successful learning

Abstract

In compressive learning, a mixture model (a set of centroids or a Gaussian mixture) is learned from a sketch vector, that serves as a highly compressed representation of the dataset. This requires solving a non-convex optimization problem, hence in practice approximate heuristics (such as CLOMPR) are used. In this work we explore, by numerical simulations, properties of this non-convex optimization landscape and those heuristics.