Stringesthesia: Dynamically Shifting Musical Agency Between Audience and Performer Based on Trust in an Interactive and Improvised Performance

TL;DR

Stringesthesia is an innovative interactive performance that uses real-time neuroimaging to dynamically shift musical agency between performers and audiences based on measured trust levels.

Contribution

The paper presents a novel neuroimaging-based system for real-time interaction in improvised performances, integrating trust metrics to influence audience participation.

Findings

Effective real-time trust measurement using fNIRS

Dynamic control of audience participation based on trust levels

Positive feedback from performers and audiences

Abstract

This paper introduces Stringesthesia, an interactive and improvised performance paradigm. Stringesthesia uses real-time neuroimaging to connect performers and audiences, enabling direct access to the performers mental state and determining audience participation during the performance. Functional near-infrared spectroscopy, or fNIRS, a noninvasive neuroimaging tool, was used to assess metabolic activity of brain areas collectively associated with a metric we call trust. A visualization representing the real-time measurement of the performers level of trust was projected behind the performer and used to dynamically restrict or promote audience participation. Throughout the paper we discuss prior work that heavily influenced our design, conceptual and methodological issues with using fNIRS technology, system architecture, and feedback from the audience and performer.