Computationally Efficient Deep Bayesian Unit-Level Modeling of Survey Data under Informative Sampling for Small Area Estimation

TL;DR

This paper introduces a computationally efficient deep Bayesian modeling approach for survey data that accounts for informative sampling, enabling uncertainty quantification and improved small area estimation.

Contribution

It presents a novel likelihood-based deep Bayesian model using random weights, incorporating pseudo-likelihood for informative sampling, and demonstrates its effectiveness through simulations and real data.

Findings

Effective uncertainty quantification in deep models

Improved small area estimation with survey data

Computational efficiency over traditional deep learning methods

Abstract

The topic of deep learning has seen a surge of interest in recent years both within and outside of the field of Statistics. Deep models leverage both nonlinearity and interaction effects to provide superior predictions in many cases when compared to linear or generalized linear models. However, one of the main challenges with deep modeling approaches is quantification of uncertainty. The use of random weight models, such as the popularized "Extreme Learning Machine," offer a potential solution in this regard. In addition to uncertainty quantification, these models are extremely computationally efficient as they do not require optimization through stochastic gradient descent, which is what is typically done for deep learning. We show how the use of random weights in a deep model can fit into a likelihood based framework to allow for uncertainty quantification of the model parameters and any desired estimates. Furthermore, we show how this approach can be used to account for informative sampling of survey data through the use of a pseudo-likelihood. We illustrate the effectiveness of this methodology through simulation and with a real survey data application involving American National Election Studies data.