Optimal integrating learning for split questionnaire design type data

TL;DR

This paper introduces an optimal integrating learning method for split questionnaire design data, effectively combining partial information to estimate regression functions with improved efficiency and reduced bias.

Contribution

The authors propose a novel averaging approach that optimally combines models from data blocks, outperforming traditional missing data methods in accuracy and computational efficiency.

Findings

The method achieves asymptotic optimality in squared loss and risk.

Simulation studies demonstrate superior performance over existing methods.

Application to European Social Survey data confirms practical effectiveness.

Abstract

In the era of data science, it is common to encounter data with different subsets of variables obtained for different cases. An example is the split questionnaire design (SQD), which is adopted to reduce respondent fatigue and improve response rates by assigning different subsets of the questionnaire to different sampled respondents. A general question then is how to estimate the regression function based on such block-wise observed data. Currently, this is often carried out with the aid of missing data methods, which may unfortunately suffer intensive computational cost, high variability, and possible large modeling biases in real applications. In this article, we develop a novel approach for estimating the regression function for SQD-type data. We first construct a list of candidate models using available data-blocks separately, and then combine the estimates properly to make an efficient use of all the information. We show the resulting averaged model is asymptotically optimal in the sense that the squared loss and risk are asymptotically equivalent to those of the best but infeasible averaged estimator. Both simulated examples and an application to the SQD dataset from the European Social Survey show the promise of the proposed method.