Assessing the credibility of a drug’s effects: identification and judgment of uncertainty by the Dutch Medicines Evaluation Board

TL;DR

This paper explores how drug regulators assess the credibility of a drug's effects by identifying uncertainties in submitted data and offers recommendations to improve the process.

Contribution

The study provides new insights into the practical methods regulators use to evaluate uncertainties in drug dossiers.

Findings

Regulators use an investigative approach to identify uncertainties in drug dossiers.

Expectations about study design and presentation influence how regulators interpret data.

Experience and expertise are crucial for evaluating uncertainties effectively.

Abstract

Medicine regulators need to judge whether a drug’s favorable effects outweigh its unfavorable effects based on a dossier submitted by an applicant, such as a pharmaceutical company. Because scientific knowledge is inherently uncertain, regulators also need to judge the credibility of these effects by identifying and evaluating uncertainties. We performed an ethnographic study of assessment procedures at the Dutch Medicines Evaluation Board (MEB) and describe how regulators evaluate the credibility of an applicant’s claims about the benefits and risks of a drug in practice. Our analysis shows that regulators use an investigative approach, which illustrates the effort required to identify uncertainties. Moreover, we show that regulators’ expectations about the presentation, the design, and the results of studies can shape how they perceive a medicine’s dossier. We highlight the importance of regulatory experience and expertise in the identification and evaluation of uncertainties. In light of our observations, we provide two recommendations to reduce avoidable uncertainty: less reliance on evidence generated by the applicant; and better communication about, and enforcement of, regulatory frameworks toward drug developers.