HAL-MLE Log-Splines Density Estimation (Part I: Univariate)

TL;DR

This paper introduces a HAL-based maximum likelihood estimator for univariate density estimation with total variation regularization, establishing its asymptotic properties and connecting it to classical TV-penalized methods.

Contribution

It provides the first theoretical analysis of the univariate HAL-MLE, proving asymptotic linearity, normality, and optimal convergence rates, linking HAL to traditional TV-penalized approaches.

Findings

HAL-MLE is asymptotically linear and normally distributed.

Achieves optimal uniform convergence rates for smoothness order k.

Connects HAL-MLE with classical TV-penalized density estimation methods.

Abstract

We study nonparametric maximum likelihood estimation of probability densities under a total variation (TV) type penalty, sectional variation norm (also named as Hardy-Krause variation). TV regularization has a long history in regression and density estimation, including results on $L^2$ and KL divergence convergence rates. Here, we revisit this task using the Highly Adaptive Lasso (HAL) framework. We formulate a HAL-based maximum likelihood estimator (HAL-MLE) using the log-spline link function from \citet{kooperberg1992logspline}, and show that in the univariate setting the bounded sectional variation norm assumption underlying HAL coincides with the classical bounded TV assumption. This equivalence directly connects HAL-MLE to existing TV-penalized approaches such as local adaptive splines \citep{mammen1997locally}. We establish three new theoretical results: (i) the univariate HAL-MLE is asymptotically linear, (ii) it admits pointwise asymptotic normality, and (iii) it achieves uniform convergence at rate $n^{-(k+1)/(2k+3)}$ up to logarithmic factors for the smoothness order $k \geq 1$. These results extend existing results from \citet{van2017uniform}, which previously guaranteed only uniform consistency without rates when $k=0$. We will include the uniform convergence for general dimension $d$ in the follow-up work of this paper. The intention of this paper is to provide a unified framework for the TV-penalized density estimation methods, and to connect the HAL-MLE to the existing TV-penalized methods in the univariate case, despite that the general HAL-MLE is defined for multivariate cases.