Photonic spiking reinforcement learning for intelligent routing

TL;DR

This paper introduces a photonic spiking reinforcement learning architecture for intelligent routing, demonstrating improved performance and energy efficiency over traditional algorithms through hardware-software integration and experimental validation.

Contribution

It presents a novel photonic spiking RL framework with hardware implementation for ultra-low latency and energy-efficient network routing, validated on SDN with a fat-tree topology.

Findings

Outperforms Dijkstra algorithm in key metrics

Hardware-software framework achieves high inference accuracy

Analyzes impact of network size and hidden layers

Abstract

Intelligent routing plays a key role in modern communication infrastructure, including data centers, computing networks, and future 6G networks. Although reinforcement learning (RL) has shown great potential for intelligent routing, its practical deployment remains constrained by high energy consumption and decision latency. Here, we propose a photonic spiking RL architecture that implements a proximal policy optimization (PPO)-based intelligent routing algorithm. The performance of the proposed approach is systematically evaluated on a software-defined network (SDN) with a fat-tree topology. The results demonstrate that, under various baseline traffic rate conditions, the PPO-based routing strategy significantly outperforms the conventional Dijkstra algorithm in several key performance metrics. Furthermore, a hardware-software collaborative framework of the spiking Actor network is realized for three typical baseline traffic rates, utilizing a photonic synapse chip based on a Mach-Zehnder interferometer (MZI) array and a photonic spiking neuron chip based on distributed feedback lasers with a saturable absorber (DFB-SAs). Experimental validation on 640 state-action pairs shows that the inference accuracy of the hardware-software collaborative framework is consistent with that of the pure algorithmic implementation. The impacts of different hidden-layer scales in the spiking Actor network and varying network size of fat-tree topology are further analyzed. The integration of photonic spiking RL with SDN-based routing establishes a novel paradigm for intelligent routing optimization, featuring ultra-low latency and high energy efficiency. This approach exhibits broad application prospects in real-time network optimization scenarios, including large-scale data centers, computing networks, satellite Internet systems, and future 6G networks.