Imputing Missing Values in the Occupational Requirements Survey

TL;DR

This paper presents a novel iterative regression-based method using XGBoost to impute missing data in the Occupational Requirements Survey, enhancing data completeness for better workforce analysis.

Contribution

Introduces a new imputation technique leveraging survey features and iterative regression with XGBoost, applicable to occupational data with missing values.

Findings

Achieves accurate imputations with 95% confidence intervals.

Enhances the utility of ORS data for workforce analysis.

Proposes a generalized imputation algorithm, WIGEM.

Abstract

The U.S. Bureau of Labor Statistics allows public access to much of the data acquired through its Occupational Requirements Survey (ORS). This data can be used to draw inferences about the requirements of various jobs and job classes within the United States workforce. However, the dataset contains a multitude of missing observations and estimates, which somewhat limits its utility. Here, we propose a method by which to impute these missing values that leverages many of the inherent features present in the survey data, such as known population limit and correlations between occupations and tasks. An iterative regression fit, implemented with a recent version of XGBoost and executed across a set of simulated values drawn from the distribution described by the known values and their standard deviations reported in the survey, is the approach used to arrive at a distribution of predicted values for each missing estimate. This allows us to calculate a mean prediction and bound said estimate with a 95% confidence interval. We discuss the use of our method and how the resulting imputations can be utilized to inform and pursue future areas of study stemming from the data collected in the ORS. Finally, we conclude with an outline of WIGEM, a generalized version of our weighted, iterative imputation algorithm that could be applied to other contexts.