Should policy makers trust composite indices? A commentary on the pitfalls of inappropriate indices for policy formation

TL;DR

This paper critically examines the limitations of global indices like the GHSI in predicting pandemic preparedness, highlighting their poor correlation with actual outcomes during COVID-19 and questioning their policy utility.

Contribution

It provides an empirical analysis showing the disconnect between index predictions and real-world pandemic performance, emphasizing the pitfalls of relying on such indices for policy decisions.

Findings

Inverse relationship between predicted and actual pandemic performance

GHSI rankings poorly correlated with COVID-19 outcomes

Indices may mislead policy responses during crises

Abstract

This paper critically discusses the use and merits of global indices, in particular, the Global Health Security Index or GHSI (Cameron et 2019) in times of an imminent crisis, like the current pandemic. The index ranked 195 countries according to their expected preparedness in case of a pandemic or other biological threat. The Covid-19 pandemic provides the background to compare each country's predicted performance from the GHSI with the actual performance. In general, there is an inverted relation between predicted versus actual performance, i.e. the predicted top performers are among those that are the worst hit. Obviously, this reflects poorly on the potential policy uses of the index in imminent crisis management. The paper also uses two different data sets, one from the Worldmeter on the spread of the Covid-19 pandemics, and the other one from the INGSA policy tracker, to make comparisons between the actual introduction of pandemic response policies and the corresponding death rate in 29 selected countries.