Network Map Reduce

TL;DR

This paper proposes a network data analytics architecture inspired by MapReduce, embedding data processing directly into network devices for real-time, efficient, and versatile network monitoring and management.

Contribution

It introduces a novel in-network MapReduce-like architecture and programming model, enabling real-time analytics directly within network infrastructure.

Findings

Enables real-time network analytics with in-network processing.

Improves cost performance by embedding analytics in network devices.

Supports a wide range of interactive network queries.

Abstract

Networking data analytics is increasingly used for enhanced network visibility and controllability. We draw the similarities between the Software Defined Networking (SDN) architecture and the MapReduce programming model. Inspired by the similarity, we suggest the necessary data plane innovations to make network data plane devices function as distributed mappers and optionally, reducers. A streaming network data MapReduce architecture can therefore conveniently solve a series of network monitoring and management problems. Unlike the traditional networking data analytical system, our proposed system embeds the data analytics engine directly in the network infrastructure. The affinity leads to a concise system architecture and better cost performance ratio. On top of this architecture, we propose a general MapReduce-like programming model for real-time and one-pass networking data analytics, which involves joint in-network and out-of-network computing. We show this model can address a wide range of interactive queries from various network applications. This position paper strives to make a point that the white-box trend does not necessarily lead to simple and dumb networking devices. Rather, the defining characteristics of the next generation white-box are open and programmable, so that the network devices can be made smart and versatile to support new services and applications.