Guided Exploration of Data Summaries

TL;DR

This paper introduces Eda4Sum, a guided, sequential approach to data summarization that improves over traditional one-shot methods by using reinforcement learning and utility maximization, especially for large, diverse datasets.

Contribution

It formalizes the Eda4Sum problem, proposing two novel methods—Top1Sum and RLSum—that enhance data summarization through guided exploration and reinforcement learning.

Findings

RLSum outperforms one-shot summarization in large datasets

Guided exploration improves diversity and utility of summaries

Reinforcement learning effectively trains summarization policies

Abstract

Data summarization is the process of producing interpretable and representative subsets of an input dataset. It is usually performed following a one-shot process with the purpose of finding the best summary. A useful summary contains k individually uniform sets that are collectively diverse to be representative. Uniformity addresses interpretability and diversity addresses representativity. Finding such as summary is a difficult task when data is highly diverse and large. We examine the applicability of Exploratory Data Analysis (EDA) to data summarization and formalize Eda4Sum, the problem of guided exploration of data summaries that seeks to sequentially produce connected summaries with the goal of maximizing their cumulative utility. EdA4Sum generalizes one-shot summarization. We propose to solve it with one of two approaches: (i) Top1Sum which chooses the most useful summary at each step; (ii) RLSum which trains a policy with Deep Reinforcement Learning that rewards an agent for finding a diverse and new collection of uniform sets at each step. We compare these approaches with one-shot summarization and top-performing EDA solutions. We run extensive experiments on three large datasets. Our results demonstrate the superiority of our approaches for summarizing very large data, and the need to provide guidance to domain experts.