Deliberation in Guesstimation

TL;DR

This paper studies how people estimate unknown quantities and finds that thinking carefully improves accuracy, though people tend to be overly confident in their answers.

Contribution

The study introduces a two-response paradigm to analyze the impact of deliberation on guesstimation accuracy and confidence.

Findings

Deliberation significantly improves the quality of guesstimation answers compared to gut feelings.

Participants are generally overconfident in their deliberated answers when asked to provide a full distribution.

Guesstimation tasks are proposed as useful for studying human deliberation and improving forecasting with AI.

Abstract

In many real‐world settings, people often have to make judgments with incomplete information. Estimating unknown quantities without using precise quantitative modeling and data is called guesstimation, which is often needed in forecasting settings. Furthermore, research in education found that solving guesstimation problems builds general problem‐solving skills. In this paper, we present an empirical investigation on how people solve guesstimation problems. We study their problem‐solving behavior with think‐aloud methods, and we identify solution strategies that are frequently used. In a two‐response paradigm, we first ask for gut‐feeling answers to guesstimation questions and then allow deliberation before a second answer is given. Comparing the quality of these two answers reveals that deliberation improves the answer quality significantly. In a second experiment, we additionally elicit participants' confidence about their deliberated answers by asking for an entire distribution instead of just a point estimate. We find that participants are generally overconfident in their answers. We discuss guesstimation tasks as suitable test‐beds for studying human deliberative judgments in general and in the more specific context of improving forecasting through appropriate artificial intelligence tools.

Figures