Control of Gene Regulatory Networks with Noisy Measurements and Uncertain Inputs

TL;DR

This paper develops a reinforcement learning-based method to control gene regulatory networks under noisy measurements and uncertain interventions, transforming the problem into a belief-space MDP and employing Gaussian processes for cost modeling.

Contribution

It introduces a novel approach combining POBDS modeling, belief-space MDP transformation, Gaussian process cost modeling, and reinforcement learning for controlling partially observed GRNs.

Findings

Effective control of large GRNs demonstrated

Near-optimal control achieved with reinforcement learning

Method validated on synthetic melanoma gene data

Abstract

This paper is concerned with the problem of stochastic control of gene regulatory networks (GRNs) observed indirectly through noisy measurements and with uncertainty in the intervention inputs. The partial observability of the gene states and uncertainty in the intervention process are accounted for by modeling GRNs using the partially-observed Boolean dynamical system (POBDS) signal model with noisy gene expression measurements. Obtaining the optimal infinite-horizon control strategy for this problem is not attainable in general, and we apply reinforcement learning and Gaussian process techniques to find a near-optimal solution. The POBDS is first transformed to a directly-observed Markov Decision Process in a continuous belief space, and the Gaussian process is used for modeling the cost function over the belief and intervention spaces. Reinforcement learning then is used to learn the cost function from the available gene expression data. In addition, we employ sparsification, which enables the control of large partially-observed GRNs. The performance of the resulting algorithm is studied through a comprehensive set of numerical experiments using synthetic gene expression data generated from a melanoma gene regulatory network.