Reply to "Comment on 'Experimentally adjudicating between different causal accounts of Bell-inequality violations via statistical model selection'"

TL;DR

This paper defends an experimental approach that compares causal models of Bell-inequality violations, clarifies misconceptions about superdeterministic models, and discusses the role of overfitting in model selection.

Contribution

It clarifies the authors' original classification of superdeterministic models and defends the use of overfitting as a criterion in model comparison.

Findings

Certain superdeterministic models overfit data relative to quantum causal models.

Overfitting is sometimes appropriate for model adjudication, contrary to claims.

The paper clarifies the distinction between different classes of superdeterministic models.

Abstract

Our article described an experiment that adjudicates between different causal accounts of Bell inequality violations by a comparison of their predictive power, finding that certain types of models that are structurally radical but parametrically conservative, of which a class of superdeterministic models are an example, overfit the data relative to models that are structurally conservative but parametrically radical in the sense of endorsing an intrinsically quantum generalization of the framework of causal modelling. In their comment (arXiv:2206.10619), Hance and Hossenfelder argue that we have misrepresented the purpose of superdeterministic models. We here dispute this claim by recalling the different classes of superdeterministic models we defined in our article and our conclusions regarding which of these are disfavoured by our experimental results. Their confusion on this point seems to have arisen in part from the fact that we characterized superdeterministic models within a causal modelling framework and from the fact that we referred to this framework as "classical" in order to contrast it with an intrinsically quantum alternative. In this reply, therefore, we take the opportunity to clarify these points. They also claim that if one is adjudicating between a pair of models, where one model can account for strictly more operational statistics than the other, the first model will tend to overfit the data relative to the second. Because this model inclusion relation can arise for pairs of models in a reductionist heirarchy, they conclude that overfitting should not be taken as evidence against the first model. We point out here that, contrary to this claim, one does not expect overfitting to arise generically in cases of model inclusion, so that it is indeed sometimes appropriate to consider overfitting as a criterion for adjudicating between such models.