Deep Reinforcement Learning for Control of Probabilistic Boolean Networks

TL;DR

This paper explores using deep reinforcement learning, specifically Double Deep Q-Networks with Prioritized Experience Replay, to control Probabilistic Boolean Networks without needing detailed network models, demonstrated on synthetic and real gene data.

Contribution

It introduces a model-free control approach for PBNs using advanced deep RL techniques, applicable to complex biological systems where network dynamics are unknown.

Findings

Effective control of PBNs demonstrated on synthetic models.

Successful application to gene-expression data from melanoma study.

Reinforcement learning outperforms traditional control methods.

Abstract

Probabilistic Boolean Networks (PBNs) were introduced as a computational model for the study of complex dynamical systems, such as Gene Regulatory Networks (GRNs). Controllability in this context is the process of making strategic interventions to the state of a network in order to drive it towards some other state that exhibits favourable biological properties. In this paper we study the ability of a Double Deep Q-Network with Prioritized Experience Replay in learning control strategies within a finite number of time steps that drive a PBN towards a target state, typically an attractor. The control method is model-free and does not require knowledge of the network's underlying dynamics, making it suitable for applications where inference of such dynamics is intractable. We present extensive experiment results on two synthetic PBNs and the PBN model constructed directly from gene-expression data of a study on metastatic-melanoma.