# High-Dimensional Learning under ApproximateSparsity with Applications to Nonsmooth Estimation and Regularized Neural Networks

## TL;DR

This paper introduces a generalized framework for high-dimensional learning that relaxes traditional assumptions, demonstrating that poly-logarithmic sample complexity suffices for nonsmooth models and neural networks, even without restricted strong convexity.

## Contribution

It extends high-dimensional statistical learning theory by relaxing sparsity and RSC conditions using folded concave penalties, enabling analysis of nonsmooth and neural network models with minimal sample complexity.

## Key findings

- Poly-logarithmic sample complexity for high-dimensional models.
- Regularization ensures generalizability of over-parameterized neural networks.
- Framework applies to nonsmooth learning problems.

## Abstract

High-dimensional statistical learning (HDSL) has wide applications in data analysis, operations research, and decision-making. Despite the availability of multiple theoretical frameworks, most existing HDSL schemes stipulate the following two conditions: (a) the sparsity, and (b) the restricted strong convexity (RSC). This paper generalizes both conditions via the use of the folded concave penalty (FCP). More specifically, we consider an M-estimation problem where (i) the (conventional) sparsity is relaxed into the approximate sparsity and (ii) the RSC is completely absent. We show that the FCP-based regularization leads to poly-logarithmic sample complexity; the training data size is only required to be poly-logarithmic in the problem dimensionality. This finding can facilitate the analysis of two important classes of models that are currently less understood: the high-dimensional nonsmooth learning and the (deep) neural networks (NN). For both problems, we show that the poly-logarithmic sample complexity can be maintained. In particular, our results indicate that the generalizability of NNs under over-parameterization can be theoretically ensured with the aid of regularization.