Visual Model Validation via Inline Replication

TL;DR

This paper introduces Inline Replication, a nonparametric validation method for visual models that enhances the repeatability and reliability of insights derived from retrospective data visualizations.

Contribution

It presents a novel validation approach called Inline Replication, adaptable to various visualization techniques and data pipelines, improving the credibility of visual predictive models.

Findings

IR provides a nonparametric validation technique for visual models.

IR can be integrated into traditional and big data visualization workflows.

Examples demonstrate IR's application in common and complex visualization systems.

Abstract

Data visualizations typically show retrospective views of an existing dataset with little or no focus on repeatability. However, consumers of these tools often use insights gleaned from retrospective visualizations as the basis for decisions about future events. In this way, visualizations often serve as visual predictive models despite the fact that they are typically designed to present historical views of the data. This "visual predictive model" approach, however, can lead to invalid inferences. In this paper, we describe an approach to visual model validation called Inline Replication (IR) which, similar to the cross-validation technique used widely in machine learning, provides a nonparametric and broadly applicable technique for visual model assessment and repeatability. This paper describes the overall IR process and outlines how it can be integrated into both traditional and emerging "big data" visualization pipelines. Examples are provided showing IR integrated within common visualization techniques (such as bar charts and linear regression lines) as well as a more fully-featured visualization system designed for complex exploratory analysis tasks.