Dependable Exploitation of High-Dimensional Unlabeled Data in an Assumption-Lean Framework

TL;DR

This paper develops a robust semi-supervised learning method that reliably improves high-dimensional regression inference by effectively utilizing unlabeled data, even under model misspecification.

Contribution

It introduces a novel estimator that guarantees at least as much efficiency as supervised methods, ensuring dependable use of unlabeled data in high-dimensional settings.

Findings

The debiased estimator's efficiency depends on the accurate estimation of the conditional mean.

The proposed estimator remains efficient even when the mean function is misspecified.

Simulation studies and real data applications demonstrate the method's effectiveness.

Abstract

Semi-supervised learning has attracted significant attention due to the proliferation of applications featuring limited labeled data but abundant unlabeled data. In this paper, we examine the statistical inference problem in an assumption-lean framework which involves a high-dimensional regression parameter, defined by minimizing the least squares, within the context of semi-supervised learning. We investigate when and how unlabeled data can enhance the estimation efficiency of a regression parameter functional. First, we demonstrate that a straightforward debiased estimator can only be more efficient than its supervised counterpart if the unknown conditional mean function can be consistently estimated at an appropriate rate. Otherwise, incorporating unlabeled data can actually be counterproductive. To address this vulnerability, we propose a novel estimator guaranteed to be at least as efficient as the supervised baseline, even when the conditional mean function is misspecified. This ensures the dependable use of unlabeled data for statistical inference. Finally, we extend our approach to the general M-estimation framework, and demonstrate the effectiveness of our methodology through comprehensive simulation studies and a real data application.