Prevalence and trend estimation from observational data with highly variable post-stratification weights

TL;DR

This paper introduces new statistical methods for estimating prevalence and trends from observational survey data with highly variable post-stratification weights, improving accuracy and stability.

Contribution

It extends weight smoothing models and GREG methods for prevalence and trend estimation, with variance estimation and validation through simulation and real data application.

Findings

Nonparametric GREG performs consistently across simulations.

Proposed methods reduce bias from variable post-stratification weights.

Application to influenza data demonstrates practical utility.

Abstract

In observational surveys, post-stratification is used to reduce bias resulting from differences between the survey population and the population under investigation. However, this can lead to inflated post-stratification weights and, therefore, appropriate methods are required to obtain less variable estimates. Proposed methods include collapsing post-strata, trimming post-stratification weights, generalized regression estimators (GREG) and weight smoothing models, the latter defined by random-effects models that induce shrinkage across post-stratum means. Here, we first describe the weight-smoothing model for prevalence estimation from binary survey outcomes in observational surveys. Second, we propose an extension of this method for trend estimation. And, third, a method is provided such that the GREG can be used for prevalence and trend estimation for observational surveys. Variance estimates of all methods are described. A simulation study is performed to compare the proposed methods with other established methods. The performance of the nonparametric GREG is consistent over all simulation conditions and therefore serves as a valuable solution for prevalence and trend estimation from observational surveys. The method is applied to the estimation of the prevalence and incidence trend of influenza-like illness using the 2010/2011 Great Influenza Survey in Flanders, Belgium.