On the Finite-Sample Analysis of $\Theta$-estimators

TL;DR

This paper provides a theoretical analysis of $ heta$-estimators, demonstrating their near-optimal statistical performance and rapid convergence in high-dimensional sparse estimation tasks, with insights into convex and nonconvex shrinkage benefits.

Contribution

It introduces a new theoretical framework for analyzing $ heta$-estimators, establishing their near-minimax optimality and fast convergence rates under regularity conditions.

Findings

Establishes oracle inequalities for $ heta$-estimators.

Shows geometric convergence to the true parameter.

Reveals benefits of convex vs. nonconvex shrinkage.

Abstract

In large-scale modern data analysis, first-order optimization methods are usually favored to obtain sparse estimators in high dimensions. This paper performs theoretical analysis of a class of iterative thresholding based estimators defined in this way. Oracle inequalities are built to show the nearly minimax rate optimality of such estimators under a new type of regularity conditions. Moreover, the sequence of iterates is found to be able to approach the statistical truth within the best statistical accuracy geometrically fast. Our results also reveal different benefits brought by convex and nonconvex types of shrinkage.