Nonparametric Regression in Dirichlet Spaces: A Random Obstacle Approach

TL;DR

This paper introduces a novel nonparametric estimation method in Dirichlet spaces that overcomes pointwise evaluation challenges by using local means, achieving rate-optimal convergence without requiring smoothness of the space.

Contribution

It develops a renormalized ridge regression technique for Dirichlet spaces, providing the first optimal out-of-sample guarantees in this general setting.

Findings

The renormalized estimator is rate-optimal.

The method applies to manifold and fractal spaces.

It does not require smoothness assumptions.

Abstract

In this paper, we consider nonparametric estimation over general Dirichlet metric measure spaces. Unlike the more commonly studied reproducing kernel Hilbert space, whose elements may be defined pointwise, a Dirichlet space typically only contain equivalence classes, i.e. its elements are only unique almost everywhere. This lack of pointwise definition presents significant challenges in the context of nonparametric estimation, for example the classical ridge regression problem is ill-posed. In this paper, we develop a new technique for renormalizing the ridge loss by replacing pointwise evaluations with certain \textit{local means} around the boundaries of obstacles centered at each data point. The resulting renormalized empirical risk functional is well-posed and even admits a representer theorem in terms of certain equilibrium potentials, which are truncated versions of the associated Green function, cut-off at a data-driven threshold. We demonstrate that the renormalized ridge estimator is rate-optimal, and derive an adaptive upper bound on its convergence rate that highlights the interplay between the analytic, geometric, and probabilistic properties of the Dirichlet form. Our framework notably does not require the smoothness of the underlying space, and is applicable to both manifold and fractal settings. To the best of our knowledge, this is the first paper to obtain optimal, out-of-sample convergence guarantees in the framework of general metric measure Dirichlet spaces.