Incorporating Expert Opinion on Observable Quantities into Statistical Models -- A General Framework

TL;DR

This paper presents a flexible framework for integrating expert opinions on observable quantities into statistical models via a loss function, updating parameters without explicit prior specification, and demonstrates its application across various models.

Contribution

The paper introduces a novel approach that incorporates expert opinions through a loss function, avoiding prior specification on parameters, and provides practical examples for implementation.

Findings

Effective integration of expert opinion improves model estimates.

The approach is straightforward to implement with existing probabilistic programming tools.

Demonstrated across survival, multivariate normal, and regression models.

Abstract

This article describes an approach to incorporate expert opinion on observable quantities through the use of a loss function which updates a prior belief as opposed to specifying parameters on the priors. Eliciting information on observable quantities allows experts to provide meaningful information on a quantity familiar to them, in contrast to elicitation on model parameters, which may be subject to interactions with other parameters or non-linear transformations before obtaining an observable quantity. The approach to incorporating expert opinion described in this paper is distinctive in that we do not specify a prior to match an expert's opinion on observed quantity, rather we obtain a posterior by updating the model parameters through a loss function. This loss function contains the observable quantity, expressed a function of the parameters, and is related to the expert's opinion which is typically operationalized as a statistical distribution. Parameters which generate observable quantities which are further from the expert's opinion incur a higher loss, allowing for the model parameters to be estimated based on their fidelity to both the data and expert opinion, with the relative strength determined by the number of observations and precision of the elicited belief. Including expert opinion in this fashion allows for a flexible specification of the opinion and in many situations is straightforward to implement with commonly used probabilistic programming software. We highlight this using three worked examples of varying model complexity including survival models, a multivariate normal distribution and a regression problem.