Random linear systems with sparse solutions -- asymptotics and large deviations

TL;DR

This paper analyzes the performance of $ ext{l}_1$ optimization in solving random under-determined linear systems with sparse solutions, focusing on phase transitions and large deviation properties to provide a comprehensive understanding of its behavior.

Contribution

It extends previous phase transition results by analyzing large deviation principles around breaking points and demonstrates the equivalence of geometric and probabilistic characterizations.

Findings

Derived explicit large deviation rate functions for $ ext{l}_1$ performance.

Established the match between geometric and probabilistic characterizations.

Provided a detailed analysis of behavior near phase transition points.

Abstract

In this paper we revisit random linear under-determined systems with sparse solutions. We consider $\ell_1$ optimization heuristic known to work very well when used to solve these systems. A collection of fundamental results that relate to its performance analysis in a statistical scenario is presented. We start things off by recalling on now classical phase transition (PT) results that we derived in \cite{StojnicCSetam09,StojnicUpper10}. As these represent the so-called breaking point characterizations, we now complement them by analyzing the behavior in a zone around the breaking points in a sense typically used in the study of the large deviation properties (LDP) in the classical probability theory. After providing a conceptual solution to these problems we attack them on a "hardcore" mathematical level attempting/hoping to be able to obtain explicit solutions as elegant as those we obtained in \cite{StojnicCSetam09,StojnicUpper10} (this time around though, the final characterizations were to be expected to be way more involved than in \cite{StojnicCSetam09,StojnicUpper10}, simply, the ultimate goals are set much higher and their achieving would provide a much richer collection of information about the $\ell_1$'s behavior). Perhaps surprisingly, the final LDP $\ell_1$ characterizations that we obtain happen to match the elegance of the corresponding PT ones from \cite{StojnicCSetam09,StojnicUpper10}. Moreover, as we have done in \cite{StojnicEquiv10}, here we also present a corresponding LDP set of results that can be obtained through an alternative high-dimensional geometry approach. Finally, we also prove that the two types of characterizations, obtained through two substantially different mathematical approaches, match as one would hope that they do.