Switching-time bioprocess control with pulse-width-modulated optogenetics

TL;DR

This paper introduces a reinforcement learning approach to optimize pulse-width modulation control of optogenetic bioprocesses, improving tunability and dynamic regulation by efficiently managing binary light inputs.

Contribution

It proposes a novel RL-based method that parametrizes control actions via duty cycle, simplifying the optimization of switching-time bioprocess control with binary inputs.

Findings

Reinforcement learning effectively optimizes pulse-width modulation control.

The duty cycle parametrization respects binary input constraints.

Enhanced process controllability through optimized light modulation.

Abstract

Biotechnology can benefit from dynamic control to improve production efficiency. In this context, optogenetics enables modulation of gene expression using light as an external input, allowing fine-tuning of protein levels to unlock dynamic metabolic control and regulation of cell growth. Optogenetic systems can be actuated by light intensity. However, relying solely on intensity-driven control (i.e., signal amplitude) may fail to properly tune optogenetic bioprocesses when the dose-response relationship (i.e., light intensity versus gene-expression strength) is steep. In these cases, tunability is effectively constrained to either fully active or fully repressed gene expression, with little intermediate regulation. Pulse-width modulation can alleviate this issue by alternating between fully ON and OFF light intensity within forcing periods, thereby smoothing the average response and enhancing process controllability. Optimizing pulse-width-modulated optogenetics entails a switching-time optimal control problem with a binary input over multiple forcing periods. While this can be formulated as a mixed-integer optimization problem on a refined control grid with monotonic input constraints, the number of decision variables can grow rapidly with increasing control-grid resolution within forcing periods and with the total number of forcing periods, complicating the task. Here, we propose an alternative solution based on reinforcement learning. We parametrize control actions via the duty cycle, a continuous proxy variable that encodes the ON-to-OFF switching time within each forcing period, thereby respecting the intrinsic binary nature of the light intensity while avoiding fine-grid binary decision variables.