Near-optimal delta-convex estimation of Lipschitz functions

TL;DR

This paper introduces a computationally feasible algorithm for estimating Lipschitz functions from noisy data, achieving near-optimal convergence rates by extending max-affine methods and employing a nonlinear feature expansion.

Contribution

It develops a novel delta-convex estimation approach that generalizes max-affine methods, incorporating adaptive partitioning and regularization to efficiently estimate Lipschitz functions.

Findings

Attains minimax convergence rate (up to logs) for Lipschitz function estimation.

Uses nonlinear feature expansion to approximate Lipschitz functions universally.

Demonstrates competitive empirical performance against existing methods.

Abstract

This paper presents a tractable algorithm for estimating an unknown Lipschitz function from noisy observations and establishes an upper bound on its convergence rate. The approach extends max-affine methods from convex shape-restricted regression to the more general Lipschitz setting. A key component is a nonlinear feature expansion that maps max-affine functions into a subclass of delta-convex functions, which act as universal approximators of Lipschitz functions while preserving their Lipschitz constants. Leveraging this property, the estimator attains the minimax convergence rate (up to logarithmic factors) with respect to the intrinsic dimension of the data under squared loss and subgaussian distributions in the random design setting. The algorithm integrates adaptive partitioning to capture intrinsic dimension, a penalty-based regularization mechanism that removes the need to know the true Lipschitz constant, and a two-stage optimization procedure combining a convex initialization with local refinement. The framework is also straightforward to adapt to convex shape-restricted regression. Experiments demonstrate competitive performance relative to other theoretically justified methods, including nearest-neighbor and kernel-based regressors.