GreyCat: Efficient What-If Analytics for Data in Motion at Scale

TL;DR

GreyCat is a scalable system that combines graphs and time series to enable efficient what-if analytics on large, continuously changing data, supporting the exploration of multiple alternative futures in real-time.

Contribution

The paper introduces GreyCat, an open source system that organizes data into Many-Worlds Graphs for scalable, real-time what-if analysis on massive, dynamic datasets.

Findings

Supports thousands of parallel worlds with millions of timestamped nodes

Efficiently forks and updates data models for real-time scenario exploration

Demonstrates scalability and effectiveness in handling complex data models

Abstract

Over the last few years, data analytics shifted from a descriptive era, confined to the explanation of past events, to the emergence of predictive techniques. Nonetheless, existing predictive techniques still fail to effectively explore alternative futures, which continuously diverge from current situations when exploring the effects of what-if decisions. Enabling prescriptive analytics therefore calls for the design of scalable systems that can cope with the complexity and the diversity of underlying data models. In this article, we address this challenge by combining graphs and time series within a scalable storage system that can organize a massive amount of unstructured and continuously changing data into multi-dimensional data models, called Many-Worlds Graphs. We demonstrate that our open source implementation, GreyCat, can efficiently fork and update thousands of parallel worlds composed of millions of timestamped nodes, such as what-if exploration.