Railgun: streaming windows for mission critical systems
Jo\~ao Oliveirinha, Ana Sofia Gomes, Pedro Cardoso, Pedro Bizarro

TL;DR
Railgun is a distributed streaming system designed for mission-critical applications, offering accurate, real-time sliding windows with low latency and high throughput, outperforming existing solutions like Flink.
Contribution
It introduces a fault-tolerant, elastic streaming system supporting real-time sliding windows for high-load, low-latency scenarios, addressing limitations of existing systems.
Findings
Lower latency than Flink in benchmarks
Low memory usage independent of window size
Supports high throughput and fault tolerance
Abstract
Some mission critical systems, such as fraud detection, require accurate, real-time metrics over long time windows on applications that demand high throughputs and low latencies. As these applications need to run "forever", cope with large and spiky data loads, they further require to be run in a distributed setting. Unsurprisingly, we are unaware of any distributed streaming system that provides all those properties. Instead, existing systems take large simplifications, such as implementing sliding windows as a fixed set of partially overlapping windows, jeopardizing metric accuracy (violating financial regulator rules) or latency (breaching service agreements). In this paper, we propose Railgun, a fault-tolerant, elastic, and distributed streaming system supporting real-time sliding windows for scenarios requiring high loads and millisecond-level latencies. We benchmarked an initial…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsCloud Computing and Resource Management · Advanced Data Storage Technologies · Peer-to-Peer Network Technologies
