Data accounting and error counting

TL;DR

This paper introduces data accounting and error counting methods that enable reverse analysis of data outputs to identify error sources, improve debugging, and enhance data pipeline development through complete data summarizations.

Contribution

It proposes a novel data summarization algebra for impact analysis, generalizes axiomatic accounting theories to data analytics, and formalizes properties for transparent impact assertions.

Findings

Complete data summarizations facilitate easier debugging.

Impact analysis helps identify minimal data changes affecting results.

Formal properties support transparent impact assertions.

Abstract

Can we infer sources of errors from outputs of the complex data analytics software? Bidirectional programming promises that we can reverse flow of software, and translate corrections of output into corrections of either input or data analysis. This allows us to achieve holy grail of automated approaches to debugging, risk reporting and large scale distributed error tracking. Since processing of risk reports and data analysis pipelines can be frequently expressed using a sequence relational algebra operations, we propose a replacement of this traditional approach with a data summarization algebra that helps to determine an impact of errors. It works by defining data analysis of a necessarily complete summarization of a dataset, possibly in multiple ways along multiple dimensions. We also present a description to better communicate how the complete summarizations of the input data may facilitates easier debugging and more efficient development of analysis pipelines. This approach can also be described as an generalization of axiomatic theories of accounting into data analytics, thus dubbed data accounting. We also propose formal properties that allow for transparent assertions about impact of individual records on the aggregated data and ease debugging by allowing to find minimal changes that change behaviour of data analysis on per-record basis.