On Periodic Reference Tracking Using Batch-Mode Reinforcement Learning with Application to Gene Regulatory Network Control

TL;DR

This paper extends batch-mode reinforcement learning, specifically Fitted Q Iteration, to solve periodic reference tracking problems by leveraging known future reference trajectories, demonstrated on gene regulatory network control.

Contribution

It introduces a novel periodic reference tracking algorithm within batch-mode reinforcement learning that explicitly uses the periodicity and future reference information.

Findings

Successfully applied to gene regulatory network control

Exploits a priori knowledge of reference trajectories

Demonstrates effectiveness in synthetic biology systems

Abstract

In this paper, we consider the periodic reference tracking problem in the framework of batch-mode reinforcement learning, which studies methods for solving optimal control problems from the sole knowledge of a set of trajectories. In particular, we extend an existing batch-mode reinforcement learning algorithm, known as Fitted Q Iteration, to the periodic reference tracking problem. The presented periodic reference tracking algorithm explicitly exploits a priori knowledge of the future values of the reference trajectory and its periodicity. We discuss the properties of our approach and illustrate it on the problem of reference tracking for a synthetic biology gene regulatory network known as the generalised repressilator. This system can produce decaying but long-lived oscillations, which makes it an interesting system for the tracking problem. In our companion paper we also take a look at the regulation problem of the toggle switch system, where the main goal is to drive the system's states to a specific bounded region in the state space.