Resource Allocation Strategies for In-Network Stream Processing

TL;DR

This paper formalizes the operator mapping problem for in-network stream processing, proving its NP-completeness and proposing polynomial heuristics evaluated through extensive simulations.

Contribution

It formalizes the operator placement problem as linear programs, proves NP-completeness, and develops polynomial heuristics for cost-effective platform design.

Findings

NP-complete nature of the operator mapping problem

Effective polynomial heuristics for operator placement

Simulation results show heuristics approach optimal solutions

Abstract

In this paper we consider the operator mapping problem for in-network stream processing applications. In-network stream processing consists in applying a tree of operators in steady-state to multiple data objects that are continually updated at various locations on a network. Examples of in-network stream processing include the processing of data in a sensor network, or of continuous queries on distributed relational databases. We study the operator mapping problem in a ``constructive'' scenario, i.e., a scenario in which one builds a platform dedicated to the application buy purchasing processing servers with various costs and capabilities. The objective is to minimize the cost of the platform while ensuring that the application achieves a minimum steady-state throughput. The first contribution of this paper is the formalization of a set of relevant operator-placement problems as linear programs, and a proof that even simple versions of the problem are NP-complete. Our second contribution is the design of several polynomial time heuristics, which are evaluated via extensive simulations and compared to theoretical bounds for optimal solutions.