Unsupervised Discovery of Behavioral Primitives from Sensorimotor Dynamic Functional Connectivity

TL;DR

This paper introduces a framework that uses dynamic functional connectivity and matrix factorization to discover primitive movement patterns from sensorimotor data in robots, aiding understanding of sensorimotor interactions.

Contribution

It presents a novel combination of mutual information, relational modeling, and matrix factorization to identify and interpret behavioral primitives from high-dimensional sensorimotor streams.

Findings

Identified sensorimotor modules and their evolving connectivity.

Decomposed connectivity patterns into interpretable motion primitives.

Framework applicable to robot learning and human movement analysis.

Abstract

The movements of both animals and robots give rise to streams of high-dimensional motor and sensory information. Imagine the brain of a newborn or the controller of a baby humanoid robot trying to make sense of unprocessed sensorimotor time series. Here, we present a framework for studying the dynamic functional connectivity between the multimodal sensory signals of a robotic agent to uncover an underlying structure. Using instantaneous mutual information, we capture the time-varying functional connectivity (FC) between proprioceptive, tactile, and visual signals, revealing the sensorimotor relationships. Using an infinite relational model, we identified sensorimotor modules and their evolving connectivity. To further interpret these dynamic interactions, we employed non-negative matrix factorization, which decomposed the connectivity patterns into additive factors and their corresponding temporal coefficients. These factors can be considered the agent's motion primitives or movement synergies that the agent can use to make sense of its sensorimotor space and later for behavior selection. In the future, the method can be deployed in robot learning as well as in the analysis of human movement trajectories or brain signals.