Could Humans Outshine AI in Visual Data Analysis?

TL;DR

This paper explores how human intuition, despite being less accurate than statistical models overall, can outperform Bayesian agents in specific scenarios like extreme samples, highlighting the value of non-normative heuristics in visual data analysis.

Contribution

It demonstrates that human intuition can outperform Bayesian models in certain visualization tasks, suggesting a potential integration of heuristics with statistical methods for better analysis.

Findings

Humans outperform Bayesian agents on extreme samples.

Participants show lower overall accuracy than statistical models.

Insights into combining human intuition with statistical analysis.

Abstract

People often use visualizations not only to explore a dataset but also to draw generalizable conclusions about underlying models or phenomena. While previous research has viewed deviations from rational analysis as problematic, we hypothesize that human reliance on non-normative heuristics may be advantageous in certain situations. In this study, we investigate scenarios where human intuition might outperform idealized statistical rationality. Our experiment assesses participants' accuracy in characterizing the parameters of known data-generating models from bivariate visualizations. Our findings show that, while participants generally demonstrated lower accuracy than statistical models, they often outperformed Bayesian agents, particularly when dealing with extreme samples. These results suggest that, even when deviating from rationality, human gut reactions to visualizations can provide an advantage. Our findings offer insights into how analyst intuition and statistical models can be integrated to improve inference and decision-making, with important implications for the design of visual analytics tools.