Deconvolution with application to estimation of sampling probabilities and the Horvitz-Thompson estimator

TL;DR

This paper presents a deconvolution method to estimate sampling probabilities and improve the Horvitz-Thompson estimator, demonstrating its effectiveness through real and simulated data examples.

Contribution

It introduces a novel deconvolution approach for estimating sampling probabilities and modifies the Horvitz-Thompson estimator accordingly.

Findings

Modified estimator performs better in examples

Effective in panel sampling and response-based sampling

Potential applications in false discovery rate estimation

Abstract

We elaborate on a deconvolution method, used to estimate the empirical distribution of unknown parameters, as suggested recently by Efron (2013). It is applied to estimating the empirical distribution of the 'sampling probabilities' of m sampled items. The estimated empirical distribution is used to modify the Horvitz-Thompson estimator. The performance of the modified Horvitz-Thompson estimator is studied in two examples. In one example the sampling probabilities are estimated based on the number of visits until a response was obtained. The other example is based on real data from panel sampling, where in four consecutive months there are corresponding four attempts to interview each member in a panel. The sampling probabilities are estimated based on the number of successful attempts. We also discuss briefly, further applications of deconvolution, including estimation of False discovery rate.