ROSE Random Forests for Robust Semiparametric Efficient Estimation

TL;DR

This paper introduces a novel random forest-based method called ROSE that achieves robust semiparametric efficiency, addressing practical challenges in estimating complex nuisance functions in semiparametric models.

Contribution

It develops a new estimator using random forests for optimal weights, providing a practical, robust alternative to traditional semiparametric efficient estimators.

Findings

ROSE estimator achieves uniform consistency across a rich class of distributions.

The method recovers a notion of robust semiparametric efficiency.

Effective in various simulated and real-world semiparametric settings.

Abstract

It is widely recognised that semiparametric efficient estimation can be hard to achieve in practice: estimators that are in theory efficient may require unattainable levels of accuracy for the estimation of complex nuisance functions. As a consequence, estimators deployed on real datasets are often chosen in a somewhat ad hoc fashion, and may suffer high variance. We study this gap between theory and practice in the context of a broad collection of semiparametric regression models that includes the generalised partially linear model. We advocate using estimators that are robust in the sense that they enjoy $\sqrt{n}$-consistent uniformly over a sufficiently rich class of distributions characterised by certain conditional expectations being estimable by user-chosen machine learning methods. We show that even asking for locally uniform estimation within such a class narrows down possible estimators to those parametrised by certain weight functions. Conversely, we show that such estimators do provide the desired uniform consistency and introduce a novel random forest-based procedure for estimating the optimal weights. We prove that the resulting estimator recovers a notion of $\textbf{ro}$bust $\textbf{s}$emiparametric $\textbf{e}$fficiency (ROSE) and provides a practical alternative to semiparametric efficient estimators. We demonstrate the effectiveness of our ROSE random forest estimator in a variety of semiparametric settings on simulated and real-world data.