# The impact of problem domain on Bayesian inferences: A systematic investigation

## TL;DR

This study investigates how different problem domains affect people's ability to make Bayesian inferences, finding that abstract problems are easier when participants lack relevant background knowledge.

## Contribution

The study provides new empirical evidence that domain content affects Bayesian reasoning, challenging the assumption that medical problems are inherently more difficult.

## Key findings

- Correct response rates were low across all domains (14.3% to 17.4%).
- Abstract problems yielded significantly higher accuracy than medical or daily-life problems.
- Performance differences suggest that background knowledge, not domain content, affects difficulty.

## Abstract

Sparse (and occasionally contradictory) evidence exists regarding the impact of domain on probabilistic updating, some of which suggests that Bayesian word problems with medical content may be especially challenging. The present research aims to address this gap in knowledge through three pre-registered online studies, which involved a total of 2,238 participants. Bayesian word problems were related to one of three domains: medical, daily-life, and abstract. In the first two cases, problems presented realistic content and plausible numerical information, while in the latter, problems contained explicitly imaginary elements. Problems across domains were matched in terms of all relevant statistical values and, as much as possible, wording. Studies 1 and 2 utilized the same set of problems, but different response elicitation methods (i.e., an open-ended and a multiple-choice question, respectively). Study 3 involved a larger number of participants per condition and a smaller set of problems to more thoroughly investigate the magnitude of differences between the domains. There was a generally low rate of correct responses (17.2%, 17.4%, and 14.3% in Studies 1, 2, and 3, respectively), consistent with accuracy levels commonly observed in the literature for this specific task with online samples. Nonetheless, a small but significant difference between domains was observed: participants’ accuracy did not differ between medical and daily-life problems, while it was significantly higher in corresponding abstract problems. These results suggest that medical problems are not inherently more difficult to solve, but rather that performance is improved with abstract problems for which participants cannot draw from their background knowledge.