Deep Reinforcement Learning for Digital Twin-Oriented Complex Networked Systems

TL;DR

This paper extends the Digital Twin Oriented Complex Networked System framework by integrating reinforcement learning to model epidemic spread, analyzing how cooperation and free-riders affect epidemic resilience and network performance.

Contribution

It introduces a temporal DT-CNS model with RL-driven nodes for epidemic simulation, incorporating different node behaviors and analyzing their impact on epidemic dynamics and network resilience.

Findings

Full cooperation improves epidemic control and network reward.

More free-riders decrease rewards and increase infections.

Higher infection rates weaken network resilience.

Abstract

The Digital Twin Oriented Complex Networked System (DT-CNS) aims to build and extend a Complex Networked System (CNS) model with progressively increasing dynamics complexity towards an accurate reflection of reality -- a Digital Twin of reality. Our previous work proposed evolutionary DT-CNSs to model the long-term adaptive network changes in an epidemic outbreak. This study extends this framework by proposeing the temporal DT-CNS model, where reinforcement learning-driven nodes make decisions on temporal directed interactions in an epidemic outbreak. We consider cooperative nodes, as well as egocentric and ignorant "free-riders" in the cooperation. We describe this epidemic spreading process with the Susceptible-Infected-Recovered ($SIR$) model and investigate the impact of epidemic severity on the epidemic resilience for different types of nodes. Our experimental results show that (i) the full cooperation leads to a higher reward and lower infection number than a cooperation with egocentric or ignorant "free-riders"; (ii) an increasing number of "free-riders" in a cooperation leads to a smaller reward, while an increasing number of egocentric "free-riders" further escalate the infection numbers and (iii) higher infection rates and a slower recovery weakens networks' resilience to severe epidemic outbreaks. These findings also indicate that promoting cooperation and reducing "free-riders" can improve public health during epidemics.