# Evaluating the Success of a Data Analysis

## TL;DR

This paper introduces a novel framework and metric for evaluating the success of data analyses based on principle alignment between analyst and audience, aiding practitioners and students.

## Contribution

It proposes a statistical model and general framework for assessing data analysis success, emphasizing principle matching over traditional metrics.

## Key findings

- Defines success as principle alignment between analyst and audience
- Introduces a statistical model for success evaluation
- Provides guidance for data science education and practice

## Abstract

A fundamental problem in the practice and teaching of data science is how to evaluate the quality of a given data analysis, which is different than the evaluation of the science or question underlying the data analysis. Previously, we defined a set of principles for describing data analyses that can be used to create a data analysis and to characterize the variation between data analyses. Here, we introduce a metric of quality evaluation that we call the success of a data analysis, which is different than other potential metrics such as completeness, validity, or honesty. We define a successful data analysis as the matching of principles between the analyst and the audience on which the analysis is developed. In this paper, we propose a statistical model and general framework for evaluating the success of a data analysis. We argue that this framework can be used as a guide for practicing data scientists and students in data science courses for how to build a successful data analysis.