Development and performance of npd for the evaluation of models with ordinal data

TL;DR

This paper extends normalized prediction distribution errors (npde) for categorical data, demonstrating their effectiveness in model evaluation and comparison through simulations and a clinical case study.

Contribution

The authors develop an adaptation of npde for categorical data and evaluate its performance against traditional methods like Chi-square tests.

Findings

npde can detect model misspecifications effectively.

Power of npde increases with sample size and probability differences.

npde provides useful graphical diagnostics in clinical categorical data analysis.

Abstract

Introduction: Normalised prediction distribution errors (npde) are used to graphically and statistically evaluate continuous responses in non-linear mixed effect models. Here, our aim was to extend npde for categorical data and to evaluate their performance. We applied our approach to a real case-study describing the evolution of severe onychomycosis (toenail infection) in a trial comparing two treatment groups. Methods: Let V denote a dataset with categorical observations. The null hypothesis H0 is that observations in V can be described by a model M. Residuals called npde can be adapted to categorical observations using jittering techniques. Their theoretical standard normal distribution can be evaluated through the Kolmogorov-Smirnov test. We evaluated the performance in terms of power through a simulation and compared it to a Chi-square. We illustrated the test and graphs on a real case-study. Results: npd were able to detect misspecifications in the structural model and model parameter value. As expected, the power to detect model misspecifications increased both with the difference in the shape of the probability, and with the sample size. Chi-square test performed better but npd could be readily applied in all type of design. Based on the toe-nail data, graphs reveal a huge discrepancy of the base model, and a good adequation for the best model we found. Conclusions: npde can be extended to categorical data, particularly in clinical settings with unbalanced design and graphs can be useful to evaluate the model as well as the covariate effects.