Striking a Balance: Evaluating How Aggregations of Multiple Forecasts Impact Judgment Under Uncertainty

TL;DR

This study evaluates how partial aggregation of multiple forecasts in visualizations affects judgment under uncertainty, finding that certain designs improve trend prediction and reduce surprise, guiding better visualization choices.

Contribution

It introduces and empirically tests three partial aggregation visualization designs, revealing their differential impacts on judgment metrics and informing visualization design for uncertainty communication.

Findings

Horizon Sampled MFV improves trend prediction

Partial aggregation reduces surprise at outcomes

Different designs suit different communication goals

Abstract

Decision-makers consult multiple forecasts to account for uncertainties when forming judgments about future events. While prior works have compared unaggregated and highly-aggregated designs for displaying multiple forecasts (e.g., Multiple Forecast Visualizations versus confidence interval plots), it remains unclear how partial aggregation impacts judgment. To investigate the effect of partial aggregation, we curated three designs that partially aggregate multiple forecasts. Through two large-scale studies (Experiment 1 n = 695 and Experiment 2 n = 389) across 14 judgment-related metrics, we observed that one design (Horizon Sampled MFV) significantly enhanced participants' ability to predict future trends, thereby reducing their surprise when confronted with the actual outcomes. Grounded in empirical evidence, we provide insights into how to design visualizations for multiple forecasts to communicate uncertainty more effectively. Specifically, since no approach excels in all metrics, we advise choosing different designs based on communication goals and prior knowledge of forecasts.