Network Structure and Biased Variance Estimation in Respondent Driven Sampling

TL;DR

This paper investigates bias in variance estimation in Respondent Driven Sampling, revealing that current methods underestimate variance due to network assumption violations, and proposes alternative estimators to address this issue.

Contribution

It identifies the critical FOM assumption in RDS variance estimation, demonstrates its violation in real networks, and proposes new estimators to reduce bias.

Findings

Current RDS variance estimators underestimate true variance.

FOM assumption is violated in all analyzed empirical networks.

Proposed estimators show potential in reducing bias.

Abstract

This paper explores bias in the estimation of sampling variance in Respondent Driven Sampling (RDS). Prior methodological work on RDS has focused on its problematic assumptions and the biases and inefficiencies of its estimators of the population mean. Nonetheless, researchers have given only slight attention to the topic of estimating sampling variance in RDS, despite the importance of variance estimation for the construction of confidence intervals and hypothesis tests. In this paper, we show that the estimators of RDS sampling variance rely on a critical assumption that the network is First Order Markov (FOM) with respect to the dependent variable of interest. We demonstrate, through intuitive examples, mathematical generalizations, and computational experiments that current RDS variance estimators will always underestimate the population sampling variance of RDS in empirical networks that do not conform to the FOM assumption. Analysis of 215 observed university and school networks from Facebook and Add Health indicates that the FOM assumption is violated in every empirical network we analyze, and that these violations lead to substantially biased RDS estimators of sampling variance. We propose and test two alternative variance estimators that show some promise for reducing biases, but which also illustrate the limits of estimating sampling variance with only partial information on the underlying population social network.