Design Principles for Data Analysis

TL;DR

This paper introduces design principles for data analysis, emphasizing the importance of thoughtful construction and workflow choices, and provides a framework for describing and teaching data analysis design.

Contribution

It proposes formal design principles for data analysis, maps them quantitatively, and offers empirical evidence of their variation among producers and consumers.

Findings

Variation in design principles among analysts and users

A formal mechanism to describe data analyses

Framework for teaching data analysis design

Abstract

The data science revolution has led to an increased interest in the practice of data analysis. While much has been written about statistical thinking, a complementary form of thinking that appears in the practice of data analysis is design thinking -- the problem-solving process to understand the people for whom a product is being designed. For a given problem, there can be significant or subtle differences in how a data analyst (or producer of a data analysis) constructs, creates, or designs a data analysis, including differences in the choice of methods, tooling, and workflow. These choices can affect the data analysis products themselves and the experience of the consumer of the data analysis. Therefore, the role of a producer can be thought of as designing the data analysis with a set of design principles. Here, we introduce design principles for data analysis and describe how they can be mapped to data analyses in a quantitative, objective and informative manner. We also provide empirical evidence of variation of principles within and between both producers and consumers of data analyses. Our work leads to two insights: it suggests a formal mechanism to describe data analyses based on the design principles for data analysis, and it provides a framework to teach students how to build data analyses using formal design principles.