Fast Data Management with Distributed Streaming SQL

TL;DR

This paper proposes a standard SQL-based streaming query model to improve fast data management in distributed stream processing systems, addressing current limitations in responsiveness and flexibility.

Contribution

It introduces a set of requirements and a novel SQL-based streaming query model for efficient management of fast data in distributed systems.

Findings

Identified key requirements for SQL-based stream querying

Proposed a standard streaming query model for fast data

Addressed responsiveness issues in current stream processing systems

Abstract

To stay competitive in today's data driven economy, enterprises large and small are turning to stream processing platforms to process high volume, high velocity, and diverse streams of data (fast data) as they arrive. Low-level programming models provided by the popular systems of today suffer from lack of responsiveness to change: enhancements require code changes with attendant large turn-around times. Even though distributed SQL query engines have been available for Big Data, we still lack support for SQL-based stream querying capabilities in distributed stream processing systems. In this white paper, we identify a set of requirements and propose a standard SQL based streaming query model for management of what has been referred to as Fast Data.