Highly Adaptive Empirical Risk Minimization with Principal Components

TL;DR

This paper introduces the PC-HA estimators, a theoretically justified dimension reduction method for Highly Adaptive Lasso, enabling practical nonparametric estimation with preserved statistical guarantees.

Contribution

It proposes the PC-HA family, providing the first principled dimension reduction for HAL with formal theoretical guarantees and transferability of asymptotic properties.

Findings

PC-HA estimators achieve dimension reduction with theoretical validity.

Score equations for PC-HA enable transfer of efficiency and asymptotic normality.

Formal results establish the statistical properties of PC-HA estimators.

Abstract

The Highly Adaptive Lasso (HAL) delivers unprecedented guarantees in nonparametric minimum loss estimation under minimal smoothness assumptions, such as dimension-free minimax optimal rates. However, the practical use of HAL has been severely limited by its exponentially growing computationally prohibitive indicator basis expansion in moderate to high dimensions. Existing screening strategies drastically reduce this dimension but lack any theoretical justification. We introduce the Principal Component Highly Adaptive (PC-HA) family of estimators, which for the first time provide a principled and theoretically valid dimension reduction. We establish formal results on the score equations solved by these PC-HA estimators, allowing to transfer plug-in efficiency and pointwise asymptotic normality results from HAL to these PC-HA estimators, under comparable complexity control.