EEG-to-F0: Establishing artificial neuro-muscular pathway for kinematics-based fundamental frequency control

TL;DR

This study demonstrates a novel neuro-muscular pathway that uses EEG signals to control vocal fundamental frequency through a biomechanical arm model, bridging neural activity and voice modulation.

Contribution

It introduces a combined machine learning and biomechanical modeling approach to decode EEG signals for F0 control, a new method linking neural activity to voice pitch regulation.

Findings

Accurately estimated F0 from EEG signals matching ground truth.

Validated the neuromuscular pathway for F0 control in a biomechanical model.

Demonstrated potential for neural-based voice modulation systems.

Abstract

The fundamental frequency (F0) of human voice is generally controlled by changing the vocal fold parameters (including tension, length and mass), which in turn is manipulated by the muscle exciters, activated by the neural synergies. In order to begin investigating the neuromuscular F0 control pathway, we simulate a simple biomechanical arm prototype (instead of an artificial vocal tract) that tends to control F0 of an artificial sound synthesiser based on the elbow movements. The intended arm movements are decoded from the EEG signal inputs (collected simultaneously with the kinematic hand data of the participant) through a combined machine learning and biomechanical modeling strategy. The machine learning model is employed to identify the muscle activation of a single-muscle arm model in ArtiSynth (from input brain signal), in order to match the actual kinematic (elbow joint angle) data . The biomechanical model utilises this estimated muscle excitation to produce corresponding changes in elbow angle, which is then linearly mapped to F0 of a vocal sound synthesiser. We use the F0 value mapped from the actual kinematic hand data (via same function) as the ground truth and compare the F0 estimated from brain signal. A detailed qualitative and quantitative performance comparison shows that the proposed neuromuscular pathway can indeed be utilised to accurately control the vocal fundamental frequency, thereby demonstrating the success of our closed loop neuro-biomechanical control scheme.