Vis Ex Machina: An Analysis of Trust in Human versus Algorithmically Generated Visualization Recommendations

TL;DR

This study investigates user trust in automated visualization recommendations, revealing initial preferences for human curation but eventual indifference, emphasizing the importance of relevance and perceived accuracy over recommendation source.

Contribution

It provides empirical insights into user trust dynamics and suggests tailoring visualization recommendations to individual user strategies.

Findings

Participants initially prefer human recommendations.

Users show indifference to recommendation source after evaluation.

Relevance and perceived accuracy are key factors in trust.

Abstract

More visualization systems are simplifying the data analysis process by automatically suggesting relevant visualizations. However, little work has been done to understand if users trust these automated recommendations. In this paper, we present the results of a crowd-sourced study exploring preferences and perceived quality of recommendations that have been positioned as either human-curated or algorithmically generated. We observe that while participants initially prefer human recommenders, their actions suggest an indifference for recommendation source when evaluating visualization recommendations. The relevance of presented information (e.g., the presence of certain data fields) was the most critical factor, followed by a belief in the recommender's ability to create accurate visualizations. Our findings suggest a general indifference towards the provenance of recommendations, and point to idiosyncratic definitions of visualization quality and trustworthiness that may not be captured by simple measures. We suggest that recommendation systems should be tailored to the information-foraging strategies of specific users.