Distributed Flow Scheduling in an Unknown Environment

TL;DR

This paper introduces a novel distributed flow scheduling algorithm for unknown environments, combining game theory and multi-armed bandit techniques to optimize network costs with proven theoretical guarantees.

Contribution

It is the first to integrate multi-armed bandit methods with distributed flow scheduling, providing an optimal regret bound and practical simulation validation.

Findings

Achieves logarithmic regret growth over time.

Ensures distributed decision-making with theoretical optimality.

Validated through simulations confirming effectiveness.

Abstract

Flow scheduling tends to be one of the oldest and most stubborn problems in networking. It becomes more crucial in the next generation network, due to fast changing link states and tremendous cost to explore the global structure. In such situation, distributed algorithms often dominate. In this paper, we design a distributed virtual game to solve the flow scheduling problem and then generalize it to situations of unknown environment, where online learning schemes are utilized. In the virtual game, we use incentives to stimulate selfish users to reach a Nash Equilibrium Point which is valid based on the analysis of the `Price of Anarchy'. In the unknown-environment generalization, our ultimate goal is the minimization of cost in the long run. In order to achieve balance between exploration of routing cost and exploitation based on limited information, we model this problem based on Multi-armed Bandit Scenario and combined newly proposed DSEE with the virtual game design. Armed with these powerful tools, we find a totally distributed algorithm to ensure the logarithmic growing of regret with time, which is optimum in classic Multi-armed Bandit Problem. Theoretical proof and simulation results both affirm this claim. To our knowledge, this is the first research to combine multi-armed bandit with distributed flow scheduling.