Algorithmic Transparency in Forecasting Support Systems

TL;DR

This paper investigates how algorithmic transparency in Forecasting Support Systems affects user adjustments, finding that transparency can reduce harmful changes but may also overwhelm users without proper training.

Contribution

It introduces and tests three FSS designs with varying transparency levels, highlighting the complex effects of transparency on forecast adjustments and user satisfaction.

Findings

Transparency reduces harmful forecast adjustments

Self-adjusting transparent components can lead to detrimental changes

Users prefer non-transparent systems without extensive training

Abstract

Most organizations adjust their statistical forecasts (e.g. on sales) manually. Forecasting Support Systems (FSS) enable the related process of automated forecast generation and manual adjustments. As the FSS user interface connects user and statistical algorithm, it is an obvious lever for facilitating beneficial adjustments whilst discouraging harmful adjustments. This paper reviews and organizes the literature on judgemental forecasting, forecast adjustments, and FSS design. I argue that algorithmic transparency may be a key factor towards better, integrative forecasting and test this assertion with three FSS designs that vary in their degrees of transparency based on time series decomposition. I find transparency to reduce the variance and amount of harmful forecast adjustments. Letting users adjust the algorithm's transparent components themselves, however, leads to widely varied and overall most detrimental adjustments. Responses indicate a risk of overwhelming users with algorithmic transparency without adequate training. Accordingly, self-reported satisfaction is highest with a non-transparent FSS.