Prediction-Powered Linear Regression: A Balance Between Interpretation and Prediction

TL;DR

This paper introduces PUMA, a framework that combines linear regression with machine learning to improve prediction accuracy while maintaining interpretability, addressing model uncertainty and selection issues.

Contribution

It presents the first joint approach to handle uncertainty from model misspecification, algorithm choice, and tuning via model averaging, with proven asymptotic optimality.

Findings

The method achieves asymptotic prediction optimality in-sample and out-of-sample.

Simulations and real data show empirical improvements over existing methods.

The approach effectively balances interpretability and predictive performance.

Abstract

Unlabeled data are increasingly prevalent in contemporary economic studies, yet their effective use for improving prediction remains challenging because the outcomes are often costly or even infeasible to observe. Machine learning methods can help label these data and achieve high predictive accuracy, but they often lack interpretability. In this paper, we propose a Prediction-powered Unified Model Averaging (PUMA) framework to combine linear regression and machine learning methods, achieving a balance between interpretation and prediction. Unlike existing works on prediction powered inference, our approach is the first to jointly address uncertainty arising from model misspecification, power-tuning selection, and the choice of machine learning algorithms by using model averaging. Theoretically, we establish the asymptotic prediction optimality of the proposed method both in-sample and out-of-sample under mild conditions, along with estimation consistency. Extensive simulations and a real-world application further demonstrate the empirical advantages of the proposed method.