DNA-SE: Towards Deep Neural-Nets Assisted Semiparametric Estimation

TL;DR

This paper introduces DNA-SE, a novel deep neural network-based framework that efficiently solves semiparametric estimation problems involving integral equations, outperforming traditional methods in scalability and accuracy.

Contribution

The paper presents the first integration of deep neural networks into semiparametric statistics to solve integral equations more efficiently and accurately.

Findings

DNA-SE outperforms traditional methods in numerical experiments.

DNA-SE demonstrates statistical advantages in real data analysis.

The framework scales well to high-dimensional problems.

Abstract

Semiparametric statistics play a pivotal role in a wide range of domains, including but not limited to missing data, causal inference, and transfer learning, to name a few. In many settings, semiparametric theory leads to (nearly) statistically optimal procedures that yet involve numerically solving Fredholm integral equations of the second kind. Traditional numerical methods, such as polynomial or spline approximations, are difficult to scale to multi-dimensional problems. Alternatively, statisticians may choose to approximate the original integral equations by ones with closed-form solutions, resulting in computationally more efficient, but statistically suboptimal or even incorrect procedures. To bridge this gap, we propose a novel framework by formulating the semiparametric estimation problem as a bi-level optimization problem; and then we develop a scalable algorithm called Deep Neural-Nets Assisted Semiparametric Estimation (DNA-SE) by leveraging the universal approximation property of Deep Neural-Nets (DNN) to streamline semiparametric procedures. Through extensive numerical experiments and a real data analysis, we demonstrate the numerical and statistical advantages of $\dnase$ over traditional methods. To the best of our knowledge, we are the first to bring DNN into semiparametric statistics as a numerical solver of integral equations in our proposed general framework.