Data Polygamy: The Many-Many Relationships among Urban Spatio-Temporal Data Sets

TL;DR

This paper introduces Data Polygamy, a scalable framework for discovering meaningful, statistically significant relationships among diverse urban spatio-temporal data sets, addressing computational and analytical challenges.

Contribution

The paper presents a novel topology-based method for efficiently identifying significant relationships across large, heterogeneous urban data sets.

Findings

Scalable analysis of over 300 urban data sets

Effective identification of meaningful relationships

Framework outperforms existing methods in scalability

Abstract

The increasing ability to collect data from urban environments, coupled with a push towards openness by governments, has resulted in the availability of numerous spatio-temporal data sets covering diverse aspects of a city. Discovering relationships between these data sets can produce new insights by enabling domain experts to not only test but also generate hypotheses. However, discovering these relationships is difficult. First, a relationship between two data sets may occur only at certain locations and/or time periods. Second, the sheer number and size of the data sets, coupled with the diverse spatial and temporal scales at which the data is available, presents computational challenges on all fronts, from indexing and querying to analyzing them. Finally, it is non-trivial to differentiate between meaningful and spurious relationships. To address these challenges, we propose Data Polygamy, a scalable topology-based framework that allows users to query for statistically significant relationships between spatio-temporal data sets. We have performed an experimental evaluation using over 300 spatial-temporal urban data sets which shows that our approach is scalable and effective at identifying interesting relationships.