Q-Learning Based Energy-Efficient Network Planning in IP-over-EON

TL;DR

This paper introduces a Q-learning based reinforcement learning approach combined with an auxiliary graph heuristic to optimize energy-efficient network planning in IP-over-EON, addressing computational challenges of large network optimization.

Contribution

It presents a novel hybrid method integrating Q-learning with an auxiliary graph heuristic for energy-efficient network planning in IP-over-EON, improving scalability and performance.

Findings

Reduces overall power consumption in network planning.

Outperforms traditional greedy heuristics in large networks.

Provides a scalable solution for complex network optimization.

Abstract

During network planning phase, optimal network planning implemented through efficient resource allocation and static traffic demand provisioning in IP-over-elastic optical network (IP-over-EON) is significantly challenging compared with the fixed-grid wavelength division multiplexing (WDM) network due to increased flexibility in IP-over-EON. Mathematical optimization models used for this purpose may not provide solution for large networks due to large computational complexity. In this regard, a greedy heuristic may be used that intuitively selects traffic elements in sequence from static traffic demand matrix and attempts to find the best solution. However, in general, such greedy heuristics offer suboptimal solutions, since appropriate traffic sequence offering the optimal performance is rarely selected. In this regard, we propose a reinforcement learning technique (in particular a Q-learning method), combined with an auxiliary graph (AG)-based energy efficient greedy method to be used for large network planning. The Q-learning method is used to decide the suitable sequence of traffic allocation such that the overall power consumption in the network reduces. In the proposed heuristic, each traffic from the given static traffic demand matrix is successively selected using the Q-learning technique and provisioned using the AG-based greedy method.