A Lightweight Algorithm to Uncover Deep Relationships in Data Tables

TL;DR

This paper introduces a lightweight, automated algorithm that uncovers deep structural relationships within tabular data, facilitating data exploration without requiring domain expertise.

Contribution

The authors present a novel, scalable forward addition algorithm that decomposes tables into layered structures to reveal hidden relationships without prior knowledge.

Findings

Successfully decomposes tables into layered structures

Scalable to tables with many columns

Provides automatic data-driven insights

Abstract

Many data we collect today are in tabular form, with rows as records and columns as attributes associated with each record. Understanding the structural relationship in tabular data can greatly facilitate the data science process. Traditionally, much of this relational information is stored in table schema and maintained by its creators, usually domain experts. In this paper, we develop automated methods to uncover deep relationships in a single data table without expert or domain knowledge. Our method can decompose a data table into layers of smaller tables, revealing its deep structure. The key to our approach is a computationally lightweight forward addition algorithm that we developed to recursively extract the functional dependencies between table columns that are scalable to tables with many columns. With our solution, data scientists will be provided with automatically generated, data-driven insights when exploring new data sets.