How to Distribute Computation in Networks

TL;DR

This paper introduces a flow-based framework for distributing computation in networks, using entropic surjectivity to analyze the limits and optimize the balance between communication and computation costs.

Contribution

It proposes a novel perspective on distributing computation by formulating a delay cost minimization problem and introducing entropic surjectivity as a key measure.

Findings

The framework links entropic surjectivity to computation processing factors.

Numerical tests demonstrate the approach on search, MapReduce, and classification tasks.

Insights into how function sparsity affects computation and communication costs.

Abstract

In network function computation is as a means to reduce the required communication flow in terms of number of bits transmitted per source symbol. However, the rate region for the function computation problem in general topologies is an open problem, and has only been considered under certain restrictive assumptions (e.g. tree networks, linear functions, etc.). In this paper, we propose a new perspective for distributing computation, and formulate a flow-based delay cost minimization problem that jointly captures the costs of communications and computation. We introduce the notion of entropic surjectivity as a measure to determine how sparse the function is and to understand the limits of computation. Exploiting Little's law for stationary systems, we provide a connection between this new notion and the computation processing factor that reflects the proportion of flow that requires communications. This connection gives us an understanding of how much a node (in isolation) should compute to communicate the desired function within the network without putting any assumptions on the topology. Our analysis characterizes the functions only via their entropic surjectivity, and provides insight into how to distribute computation. We numerically test our technique for search, MapReduce, and classification tasks, and infer for each task how sensitive the processing factor to the entropic surjectivity is.