Probabilistic inference when the population space is open

TL;DR

This paper proposes a novel approach for probabilistic inference in open population spaces, where the sampling distribution is not confined to a fixed parametric family, by partitioning the space and using models of the population rather than sampling distributions.

Contribution

It introduces a method to perform inference without assuming a closed population space, using partitions and models of the population instead of fixed sampling distributions.

Findings

Partitioning the population space allows for flexible inference.

Using models of the population rather than sampling distributions is effective.

Standard P values are deemed neither meaningful nor useful in this context.

Abstract

In using observed data to make inferences about a population quantity, it is commonly assumed that the sampling distribution from which the data were drawn belongs to a given parametric family of distributions, or at least, a given finite set of such families, i.e. the population space is assumed to be closed. Here, we address the problem of how to determine an appropriate post-data distribution for a given population quantity when such an assumption about the underlying sampling distribution is not made, i.e. when the population space is open. The strategy used to address this problem is based on the fact that even though, due to an open population space being non-measurable, we are not able to place a post-data distribution over all the sampling distributions contained in such a population space, it is possible to partition this type of space into a finite, countable or uncountable number of subsets such that a distribution can be placed over a variable that simply indicates which of these subsets contains the true sampling distribution. Moreover, it is argued that, by using sampling distributions that belong to a number of parametric families, it is possible to adequately and elegantly represent the sampling distributions that belong to each of the subsets of such a partition. Since a statistical model is conceived as being a model of a population space rather than a model of a sampling distribution, it is also argued that neither the type of models that are put forward nor the expression of pre-data knowledge via such models can be directly brought into question by the data. Finally, the case is made that, as well as not being required in the modelling process that is proposed, the standard practice of using P values to measure the absolute compatibility of an individual or family of sampling distributions with observed data is neither meaningful nor useful.