Mesh-Based Solutions for Nonparametric Penalized Regression

TL;DR

This paper introduces a mesh-based approximation method for nonparametric penalized regression, transforming complex functional problems into finite convex optimization tasks, enabling efficient computation and maintaining statistical optimality.

Contribution

The authors propose a novel mesh-based solution (MBS) for nonparametric penalized regression that simplifies intractable problems into finite convex minimizations, facilitating practical computation.

Findings

MBS effectively approximates NPR in various regression scenarios.

The method maintains rate-optimality with increasing sample size.

An efficient algorithm leverages sparsity in MBS for faster computation.

Abstract

It is often of interest to estimate regression functions non-parametrically. Penalized regression (PR) is one statistically-effective, well-studied solution to this problem. Unfortunately, in many cases, finding exact solutions to PR problems is computationally intractable. In this manuscript, we propose a mesh-based approximate solution (MBS) for those scenarios. MBS transforms the complicated functional minimization of NPR, to a finite parameter, discrete convex minimization; and allows us to leverage the tools of modern convex optimization. We show applications of MBS in a number of explicit examples (including both uni- and multi-variate regression), and explore how the number of parameters must increase with our sample-size in order for MBS to maintain the rate-optimality of NPR. We also give an efficient algorithm to minimize the MBS objective while effectively leveraging the sparsity inherent in MBS.