Learning-Based Adaptive Dynamic Routing with Stability Guarantee for a Single-Origin-Single-Destination Network

TL;DR

This paper introduces a learning-based adaptive routing method for single-origin-single-destination networks that guarantees stability and efficiently learns near-optimal policies using a piecewise-linear function and policy iteration.

Contribution

It develops a stability-guaranteed adaptive routing algorithm using a piecewise-linear function integrated into Lyapunov analysis and policy iteration, improving computational efficiency.

Findings

The proposed method ensures mean boundedness of traffic state.

It learns near-optimal routing policies with high efficiency.

The approach outperforms neural network-based algorithms in computational speed.

Abstract

We consider learning-based adaptive dynamic routing for a single-origin-single-destination queuing network with stability guarantees. Specifically, we study a class of generalized shortest path policies that can be parameterized by only two constants via a piecewise-linear function. Using the Foster-Lyapunov stability theory, we develop a criterion on the parameters to ensure mean boundedness of the traffic state. Then, we develop a policy iteration algorithm that learns the parameters from realized sample paths. Importantly, the piecewise-linear function is both integrated into the Lyapunov function for stability analysis and used as a proxy of the value function for policy iteration; hence, stability is inherently ensured for the learned policy. Finally, we demonstrate via a numerical example that the proposed algorithm learns a near-optimal routing policy with an acceptable optimality gap but significantly higher computational efficiency compared with a standard neural network-based algorithm.