Fast Data: Moving beyond from Big Data's map-reduce

TL;DR

This paper proposes a new data processing model that moves beyond traditional map-reduce paradigms, focusing on flow-oriented functions suitable for time-dependent problems, challenging the suitability of Big Data approaches for all scenarios.

Contribution

It introduces a flow-oriented data model as an alternative to map-reduce, addressing limitations of Big Data methods for certain time-sensitive applications.

Findings

The new model improves efficiency for time-dependent data processing.

Flow-oriented functions can replace traditional map-reduce in specific contexts.

The approach offers a more suitable solution for cache-bound and time-sensitive problems.

Abstract

Big Data may not be the solution many are looking for. The latest rise of Big Data methods and systems is partly due to the new abilities these techniques provide, partly to the simplicity of the software design and partly because the buzzword itself has value to investors and clients. That said, popularity is not a measure for suitability and the Big Data approach might not be the best solution, or even an applicable one, to many common problems. Namely, time dependent problems whose solution may be bound or cached in any manner can benefit greatly from moving to partly stateless, flow oriented functions and data models. This paper presents such a model to substitute the traditional map-shuffle-reduce models.