Interactive Data Exploration with Smart Drill-Down

TL;DR

This paper introduces smart drill-down, an interactive operator for exploring relational tables by discovering and summarizing interesting groups of data through rules, with algorithms for efficient approximation and sampling.

Contribution

The paper proposes the smart drill-down operator, formulates the underlying optimization as NP-hard, and provides approximation algorithms and sampling methods for effective data exploration.

Findings

Algorithms effectively approximate optimal rule lists.

Sampling scheme enables efficient interaction with large tables.

Experiments demonstrate usefulness and performance of the approach.

Abstract

We present {\em smart drill-down}, an operator for interactively exploring a relational table to discover and summarize "interesting" groups of tuples. Each group of tuples is described by a {\em rule}. For instance, the rule $(a, b, \star, 1000)$ tells us that there are a thousand tuples with value $a$ in the first column and $b$ in the second column (and any value in the third column). Smart drill-down presents an analyst with a list of rules that together describe interesting aspects of the table. The analyst can tailor the definition of interesting, and can interactively apply smart drill-down on an existing rule to explore that part of the table. We demonstrate that the underlying optimization problems are {\sc NP-Hard}, and describe an algorithm for finding the approximately optimal list of rules to display when the user uses a smart drill-down, and a dynamic sampling scheme for efficiently interacting with large tables. Finally, we perform experiments on real datasets on our experimental prototype to demonstrate the usefulness of smart drill-down and study the performance of our algorithms.