Causal Coupled Mechanisms: A Control Method with Cooperation and Competition for Complex System

TL;DR

This paper introduces Causal Coupled Mechanisms (CCMs), a hierarchical reinforcement learning-based control method that divides complex systems into modules with cooperation and competition, achieving robust and generalizable control in noisy and varied environments.

Contribution

The paper proposes a novel control framework combining cooperation and competition in modular structures using hierarchical reinforcement learning, inspired by human cognition.

Findings

Achieves robust control in synthetic and biological systems with noise.

Demonstrates superior performance and generalization across different environments.

Reusing specialized CCMs enhances adaptability to changing dynamics.

Abstract

Complex systems are ubiquitous in the real world and tend to have complicated and poorly understood dynamics. For their control issues, the challenge is to guarantee accuracy, robustness, and generalization in such bloated and troubled environments. Fortunately, a complex system can be divided into multiple modular structures that human cognition appears to exploit. Inspired by this cognition, a novel control method, Causal Coupled Mechanisms (CCMs), is proposed that explores the cooperation in division and competition in combination. Our method employs the theory of hierarchical reinforcement learning (HRL), in which 1) the high-level policy with competitive awareness divides the whole complex system into multiple functional mechanisms, and 2) the low-level policy finishes the control task of each mechanism. Specifically for cooperation, a cascade control module helps the series operation of CCMs, and a forward coupled reasoning module is used to recover the coupling information lost in the division process. On both synthetic systems and a real-world biological regulatory system, the CCM method achieves robust and state-of-the-art control results even with unpredictable random noise. Moreover, generalization results show that reusing prepared specialized CCMs helps to perform well in environments with different confounders and dynamics.