Cumulative differences between subpopulations versus body mass index in the Behavioral Risk Factor Surveillance System data

TL;DR

This paper applies cumulative difference statistics to analyze body mass index data from the BRFSS, demonstrating their effectiveness in assessing calibration and subpopulation response differences, with methodological extensions for uncertainty estimation and treatment effect analysis.

Contribution

It introduces new empirical estimators for uncertainty and weighted treatment effects in cumulative difference analysis, extending prior methods to biostatistics data.

Findings

Cumulative difference methods effectively assess calibration in BRFSS data.

New estimators provide valid uncertainty measures for any real-valued responses.

Analysis confirms prior findings and extends methodology for biostatistical applications.

Abstract

Prior works have demonstrated many advantages of cumulative statistics over the classical methods of reliability diagrams, ECEs (empirical, estimated, or expected calibration errors), and ICIs (integrated calibration indices). The advantages pertain to assessing calibration of predicted probabilities, comparison of responses from a subpopulation to the responses from the full population, and comparison of responses from one subpopulation to those from a separate subpopulation. The cumulative statistics include graphs of cumulative differences as a function of the scalar covariate, as well as metrics due to Kuiper and to Kolmogorov and Smirnov that summarize the graphs into single scalar statistics and associated P-values (also known as "attained significance levels" for significance tests). However, the prior works have not yet treated data from biostatistics. Fortunately, the advantages of the cumulative statistics extend to the Behavioral Risk Factor Surveillance System (BRFSS) of the Centers for Disease Control and Prevention. This is unsurprising, since the mathematics is the same as in earlier works. Nevertheless, detailed analysis of the BRFSS data is revealing and corroborates the findings of earlier works. Two methodological extensions beyond prior work that facilitate analysis of the BRFSS are (1) empirical estimators of uncertainty for graphs of the cumulative differences between two subpopulations, such that the estimators are valid for any real-valued responses, and (2) estimators of the weighted average treatment effect for the differences in the responses between the subpopulations. Both of these methods concern the case in which none of the covariate's observed values for one subpopulation is equal to any of the covariate's values for the other subpopulation. The data analysis presented reports results for this case as well as several others.