Purely Bayesian counterfactuals versus Newcomb's paradox

TL;DR

This paper distinguishes between an entity's epistemic and decision systems, analyzing Bayesian counterfactuals and Newcomb's paradox, and shows how beliefs about data influence optimal strategies.

Contribution

It introduces a formal separation of epistemic and decision systems and analyzes how beliefs about data affect counterfactual decision-making in Newcomb-like problems.

Findings

Bayesian counterfactuals are estimated by the epistemic system, not the decision system.

The counterfactual optimality of 1-Box or 2-Box strategies depends on the prior beliefs about the predictor’s data.

Entities' decision-making can be better understood by separating epistemic, decision, data collection, reward, and maintenance systems.

Abstract

This paper proposes a careful separation between an entity's epistemic system and their decision system. Crucially, Bayesian counterfactuals are estimated by the epistemic system; not by the decision system. Based on this remark, I prove the existence of Newcomb-like problems for which an epistemic system necessarily expects the entity to make a counterfactually bad decision. I then address (a slight generalization of) Newcomb's paradox. I solve the specific case where the player believes that the predictor applies Bayes rule with a supset of all the data available to the player. I prove that the counterfactual optimality of the 1-Box strategy depends on the player's prior on the predictor's additional data. If these additional data are not expected to reduce sufficiently the predictor's uncertainty on the player's decision, then the player's epistemic system will counterfactually prefer to 2-Box. But if the predictor's data is believed to make them quasi-omniscient, then 1-Box will be counterfactually preferred. Implications of the analysis are then discussed. More generally, I argue that, to better understand or design an entity, it is useful to clearly separate the entity's epistemic, decision, but also data collection, reward and maintenance systems, whether the entity is human, algorithmic or institutional.