Adaptive Estimation In High-Dimensional Additive Models With Multi-Resolution Group Lasso

TL;DR

This paper introduces a multi-resolution group Lasso method for high-dimensional additive models that adaptively achieves optimal error bounds without prior knowledge of sparsity or smoothness, applicable to random designs.

Contribution

It proposes a unified adaptive estimation method that attains or surpasses existing error bounds without needing prior sparsity or smoothness information.

Findings

Achieves adaptive convergence rates under certain conditions.

Provides bounds on the prediction factor for random designs.

Works with nearly optimal sample sizes.

Abstract

In additive models with many nonparametric components, a number of regularized estimators have been proposed and proven to attain various error bounds under different combinations of sparsity and fixed smoothness conditions. Some of these error bounds match minimax rates in the corresponding settings. Some of the rate minimax methods are non-convex and computationally costly. From these perspectives, the existing solutions to the high-dimensional additive nonparametric regression problem are fragmented. In this paper, we propose a multi-resolution group Lasso (MR-GL) method in a unified approach to simultaneously achieve or improve existing error bounds and provide new ones without the knowledge of the level of sparsity or the degree of smoothness of the unknown functions. Such adaptive convergence rates are established when a prediction factor can be treated as a constant. Furthermore, we prove that the prediction factor, which can be bounded in terms of a restricted eigenvalue or a compatibility coefficient, can be indeed treated as a constant for random designs under a nearly optimal sample size condition.