Progressive Analytics: A Computation Paradigm for Exploratory Data Analysis

TL;DR

Progressive Analytics introduces a new computation paradigm that enables low-latency, progressive data analysis at the programming language level, facilitating scalable and interactive exploratory data analysis systems.

Contribution

It proposes a novel paradigm and prototype implementation that simplify building scalable, low-latency exploratory data analysis tools by integrating progressive computation into programming languages.

Findings

Prototype system ProgressiVis demonstrates low-latency data exploration.

Progressive computation reduces implementation complexity for scalable analysis.

The paradigm supports interactive, real-time feedback during data exploration.

Abstract

Exploring data requires a fast feedback loop from the analyst to the system, with a latency below about 10 seconds because of human cognitive limitations. When data becomes large or analysis becomes complex, sequential computations can no longer be completed in a few seconds and data exploration is severely hampered. This article describes a novel computation paradigm called Progressive Computation for Data Analysis or more concisely Progressive Analytics, that brings at the programming language level a low-latency guarantee by performing computations in a progressive fashion. Moving this progressive computation at the language level relieves the programmer of exploratory data analysis systems from implementing the whole analytics pipeline in a progressive way from scratch, streamlining the implementation of scalable exploratory data analysis systems. This article describes the new paradigm through a prototype implementation called ProgressiVis, and explains the requirements it implies through examples.