QUIS: Question-guided Insights Generation for Automated Exploratory Data Analysis

TL;DR

QUIS is a fully automated system for exploratory data analysis that generates and refines questions to produce relevant insights without human input or extensive retraining.

Contribution

It introduces a novel two-stage approach with question generation and insight analysis, eliminating the need for human-curated data and enabling adaptation to new datasets.

Findings

Operates fully automatically without human intervention.

Refines questions iteratively to improve insight coverage.

Adapts to new datasets without retraining.

Abstract

Discovering meaningful insights from a large dataset, known as Exploratory Data Analysis (EDA), is a challenging task that requires thorough exploration and analysis of the data. Automated Data Exploration (ADE) systems use goal-oriented methods with Large Language Models and Reinforcement Learning towards full automation. However, these methods require human involvement to anticipate goals that may limit insight extraction, while fully automated systems demand significant computational resources and retraining for new datasets. We introduce QUIS, a fully automated EDA system that operates in two stages: insight generation (ISGen) driven by question generation (QUGen). The QUGen module generates questions in iterations, refining them from previous iterations to enhance coverage without human intervention or manually curated examples. The ISGen module analyzes data to produce multiple relevant insights in response to each question, requiring no prior training and enabling QUIS to adapt to new datasets.