Pan-disease clustering analysis of the trend of period prevalence

TL;DR

This study introduces a novel clustering method to analyze the joint prevalence trends of multiple diseases over time, revealing meaningful disease groupings from Taiwan's extensive health data.

Contribution

It develops a new penalization pursuit approach for pan-disease clustering of prevalence trends, applied to large-scale national health data.

Findings

Identified 35 disease clusters with similar prevalence trends

Discovered significant differences from alternative clustering methods

Provided interpretable disease groupings with sound clinical relevance

Abstract

For all diseases, prevalence has been carefully studied. In the "classic" paradigm, the prevalence of different diseases has usually been studied separately. Accumulating evidences have shown that diseases can be "correlated". The joint analysis of prevalence of multiple diseases can provide important insights beyond individual-disease analysis, however, has not been well conducted. In this study, we take advantage of the uniquely valuable Taiwan National Health Insurance Research Database (NHIRD), and conduct a pan-disease analysis of period prevalence trend. The goal is to identify clusters within which diseases share similar period prevalence trends. For this purpose, a novel penalization pursuit approach is developed, which has an intuitive formulation and satisfactory properties. In data analysis, the period prevalence values are computed using records on close to 1 million subjects and 14 years of observation. For 405 diseases, 35 nontrivial clusters (with sizes larger than one) and 27 trivial clusters (with sizes one) are identified. The results differ significantly from those of the alternatives. A closer examination suggests that the clustering results have sound interpretations. This study is the first to conduct a pan-disease clustering analysis of prevalence trend using the uniquely valuable NHIRD data and can have important value in multiple aspects.