A first realization of reinforcement learning-based closed-loop EEG-TMS

TL;DR

This paper demonstrates a novel reinforcement learning-based closed-loop EEG-TMS system that identifies individual brain states in real-time, enabling personalized neurostimulation and advancing treatment options for brain disorders.

Contribution

It introduces the first machine-learning-based closed-loop EEG-TMS setup that automatically detects individual mu-rhythm phases linked to excitability states without user-defined targets.

Findings

Reinforcement learning accurately identified high- and low-excitability mu-rhythm phases.

Repetitive stimulation led to long-term changes in functional connectivity.

The system is feasible for real-time, individualized brain stimulation.

Abstract

Background: Transcranial magnetic stimulation (TMS) is a powerful tool to investigate neurophysiology of the human brain and treat brain disorders. Traditionally, therapeutic TMS has been applied in a one-size-fits-all approach, disregarding inter- and intra-individual differences. Brain state-dependent EEG-TMS, such as coupling TMS with a pre-specified phase of the sensorimotor mu-rhythm, enables the induction of differential neuroplastic effects depending on the targeted phase. But this approach is still user-dependent as it requires defining an a-priori target phase. Objectives: To present a first realization of a machine-learning-based, closed-loop real-time EEG-TMS setup to identify user-independently the individual mu-rhythm phase associated with high- vs. low-corticospinal excitability states. Methods: We applied EEG-TMS to 25 participants targeting the supplementary motor area-primary motor cortex network and used a reinforcement learning algorithm to identify the mu-rhythm phase associated with high- vs. low corticospinal excitability. We employed linear mixed effects models and Bayesian analysis to determine effects of reinforced learning on corticospinal excitability indexed by motor evoked potential amplitude, and functional connectivity indexed by the imaginary part of resting-state EEG coherence. Results: Reinforcement learning effectively identified the mu-rhythm phase associated with high- vs. low-excitability states, and their repetitive stimulation resulted in long-term increases vs. decreases in functional connectivity in the stimulated sensorimotor network. Conclusions: We demonstrated for the first time the feasibility of closed-loop EEG-TMS in humans, a critical step towards individualized treatment of brain disorders.