A Novel Switch-Type Policy Network for Resource Allocation Problems: Technical Report

TL;DR

This paper introduces a switch-type neural network architecture that enhances the efficiency and generalization of deep reinforcement learning policies for queueing network control, outperforming traditional MLPs especially in unseen environments.

Contribution

The paper presents the novel switch-type neural network (STN) architecture, demonstrating its superior sample efficiency and generalization over MLPs in queueing network control tasks.

Findings

STNs achieve better sample efficiency than MLPs.

STNs match MLP performance in familiar environments.

STNs outperform MLPs in new, unseen environments.

Abstract

Deep Reinforcement Learning (DRL) has become a powerful tool for developing control policies in queueing networks, but the common use of Multi-layer Perceptron (MLP) neural networks in these applications has significant drawbacks. MLP architectures, while versatile, often suffer from poor sample efficiency and a tendency to overfit training environments, leading to suboptimal performance on new, unseen networks. In response to these issues, we introduce a switch-type neural network (STN) architecture designed to improve the efficiency and generalization of DRL policies in queueing networks. The STN leverages structural patterns from traditional non-learning policies, ensuring consistent action choices across similar states. This design not only streamlines the learning process but also fosters better generalization by reducing the tendency to overfit. Our works presents three key contributions: first, the development of the STN as a more effective alternative to MLPs; second, empirical evidence showing that STNs achieve superior sample efficiency in various training scenarios; and third, experimental results demonstrating that STNs match MLP performance in familiar environments and significantly outperform them in new settings. By embedding domain-specific knowledge, the STN enhances the Proximal Policy Optimization (PPO) algorithm's effectiveness without compromising performance, suggesting its suitability for a wide range of queueing network control problems.