Information-theoretic evaluation of covariate distributions models

TL;DR

This paper evaluates covariate distribution models using an information-theoretic measure, demonstrating the advantages of non-Gaussian models like copulas and MICE in life science data, and introduces a new confidence interval construction for KL divergence.

Contribution

It provides a systematic comparison of covariate distribution models with a novel method for confidence interval estimation of KL divergence, highlighting their strengths and limitations.

Findings

Non-Gaussian models outperform Gaussian in KL-D across datasets.

Copula models generalize well to new data, MICE tends to overfit.

Parametric copulas and MICE scale better with data dimension.

Abstract

Statistical modelling of covariate distributions allows to generate virtual populations or to impute missing values in a covariate dataset. Covariate distributions typically have non-Gaussian margins and show nonlinear correlation structures, which simple multivariate Gaussian distributions fail to represent. Prominent non-Gaussian frameworks for covariate distribution modelling are copula-based models and models based on multiple imputation by chained equations (MICE). While both frameworks have already found applications in the life sciences, a systematic investigation of their goodness-of-fit to the theoretical underlying distribution, indicating strengths and weaknesses under different conditions, is still lacking. To bridge this gap, we thoroughly evaluated covariate distribution models in terms of Kullback-Leibler divergence (KL-D), a scale-invariant information-theoretic goodness-of-fit criterion for distributions. Methodologically, we proposed a new approach to construct confidence intervals for KL-D by combining nearest neighbour-based KL-D estimators with subsampling-based uncertainty quantification. In relevant data sets of different sizes and dimensionalities with both continuous and discrete covariates, non-Gaussian models showed consistent improvements in KL-D, compared to simpler Gaussian or scale transform approximations. KL-D estimates were also robust to the inclusion of latent variables and large fractions of missing values. While good generalization behaviour to new data could be seen in copula-based models, MICE shows a trend for overfitting and its performance should always be evaluated on separate test data. Parametric copula models and MICE were found to scale much better with the dataset dimension than nonparametric copula models. These findings corroborate the potential of non-Gaussian models for modelling realistic life science covariate distributions.