Asymmetric Tobit analysis for correlation estimation from censored data

TL;DR

This paper introduces an asymmetric Tobit model that leverages domain knowledge to improve correlation estimation from censored water contamination data, addressing low detection issues and enhancing imputation accuracy.

Contribution

The paper develops a novel asymmetric Tobit model that incorporates domain knowledge for better correlation estimation in censored datasets, specifically for water microbial contamination.

Findings

Imputation accuracy improves with domain knowledge integration.

The asymmetric Tobit model outperforms traditional methods in correlation estimation.

Empirical results validate the effectiveness of the proposed approach.

Abstract

Contamination of water resources with pathogenic microorganisms excreted in human feces is a worldwide public health concern. Surveillance of fecal contamination is commonly performed by routine monitoring for a single type or a few types of microorganism(s). To design a feasible routine for periodic monitoring and to control risks of exposure to pathogens, reliable statistical algorithms for inferring correlations between concentrations of microorganisms in water need to be established. Moreover, because pathogens are often present in low concentrations, some contaminations are likely to be under a detection limit. This yields a pairwise left-censored dataset and complicates computation of correlation coefficients. Errors of correlation estimation can be smaller if undetected values are imputed better. To obtain better imputations, we utilize side information and develop a new technique, the \emph{asymmetric Tobit model} which is an extension of the Tobit model so that domain knowledge can be exploited effectively when fitting the model to a censored dataset. The empirical results demonstrate that imputation with domain knowledge is effective for this task.