Engineering Music to Slow Breathing and Invite Relaxed Physiology

TL;DR

This study developed an interactive music system that uses biometric data to slow breathing and promote relaxation, demonstrating its effectiveness even when users are engaged in demanding tasks.

Contribution

The paper introduces a novel biometric-driven interactive music system that effectively reduces breathing rate and physiological arousal, surpassing traditional music interventions.

Findings

Personalized music designs significantly slowed breathing rates.

Physiological arousal indicators decreased during music interaction.

Personalized tempo had the strongest calming effect.

Abstract

We engineered an interactive music system that influences a user's breathing rate to induce a relaxation response. This system generates ambient music containing periodic shifts in loudness that are determined by the user's own breathing patterns. We evaluated the efficacy of this music intervention for participants who were engaged in an attention-demanding task, and thus explicitly not focusing on their breathing or on listening to the music. We measured breathing patterns in addition to multiple peripheral and cortical indicators of physiological arousal while users experienced three different interaction designs: (1) a "Fixed Tempo" amplitude modulation rate at six beats per minute; (2) a "Personalized Tempo" modulation rate fixed at 75\% of each individual's breathing rate baseline, and (3) a "Personalized Envelope" design in which the amplitude modulation matches each individual's breathing pattern in real-time. Our results revealed that each interactive music design slowed down breathing rates, with the "Personalized Tempo" design having the largest effect, one that was more significant than the non-personalized design. The physiological arousal indicators (electrodermal activity, heart rate, and slow cortical potentials measured in EEG) showed concomitant reductions, suggesting that slowing users' breathing rates shifted them towards a more calmed state. These results suggest that interactive music incorporating biometric data may have greater effects on physiology than traditional recorded music.

Figures

Equations

$$z_{IRI}=\frac{IRI-{\mu }_{IRI}}{{\sigma }_{IRI}}$$

$$slop{e}_{z_{EDA}}=fs\times (z_{EDA[2:n]}-z_{EDA[1:n-1]})$$

Full text

I Introduction

Music invites strong emotional responses in its listeners, and thus presents a promising avenue for the design of interaction systems that bring health and well-being to users. Historically, researchers in the music cognition community have attributed musical emotion to cognitive appraisal [1]; however, more recent models acknowledge the possibility of multiple mechanisms by which music evokes emotions [2, 3], including brain stem reflexes and emotional contagion [4]. In fact, listeners have reported many different physical reactions due to focused music listening [5]. Music has been shown to help alleviate pain [6], influence movement synchronization [7, 8], and be used as a bio-feedback signal [9]. In the present research, we explore the possibility that indirect exposure to musical stimuli may also affect users in such a way as to produce beneficial changes in their physiology, without requiring attentive listening.

Past research has detailed the range of both perceived and felt emotions that accompany attentive music listening [10], in addition to a range of physical [11] and physiological [12][13] responses associated with emotions. However, less is known about how music may be specifically designed to induce a particular physiological response in the listener. Previous studies have shown that influencing breathing may be one effective pathway to inviting shifts in physiology. For example, instructing users with music in the control of their breathing patterns showed promise in breath regulation [14], relaxation [15], creating a meditative experience [16], and reducing muscle tension [17]. However, such studies relied on intentional breathing to be effective. Other studies have shown that listening to music can effect breathing rates [18], introducing white noise to music can influence electrodermal activity and heart rate variability [19], though these were not carefully controlled to avoid intentional manipulation of breathing. The present study was carefully designed to examine any effects that auditory feedback may have on breathing even when the listener is unaware that an intervention is taking place, remaining fully engaged in a demanding task.

Breathing is a promising avenue to explore for regulation of affective state, as it is a physiological process that is under autonomic control, yet which can also be controlled externally through conscious effort or external influences [20]. The rate and manner of one’s breathing pattern changes with physical exertion [21] and affective state [22, 23]; in turn, aerobic capacity, stress level [24, 25], mental functions [26], and mood [27] can be influenced by consciously manipulating breathing, as is detailed in ancient Yoga Sutras [28], among more modern scientific research.

It is known that reducing one’s breathing rate can reduce perceived momentary stress levels [24, 25]. While consciously slowing down breathing using mindfulness exercises can be effective [29], such interventions require focused attention and divert attention from important tasks [30], which can make them impractical in workplaces. Researchers are beginning to explore implicit bio-feedback[31]. Recent findings showed that rhythmically oscillating audiovisual feedback presented to users significantly influenced slower breathing, had a lasting effect, improved self-reported calmness and focus, and was highly preferred for future use [32]. To this end, we designed three interactive music systems incorporating rhythmic loudness changes, also using a breath sensor (Zephyr BioHarness), to probe which interactions optimally, yet effortlessly, influence breathing rate through sound.

II Theory and Design

This interactive music system was designed to influence the user’s breathing pattern in order to invite a relaxation response. It was engineered with two competing principles in mind: first, this system must be as unobtrusive as possible, so as to not require focused attention that would detract from everyday activities or workplace tasks. We also wanted to avoid any prior familiarity with the music that might have prior emotional associations. Second, this system must invite physiological changes in the listener that accompany a relaxed state. To satisfy the first constraints, we engineered an original ambient-genre music composition using the PureData (Pd)[33] graphical programming language. Our software took an incoming stream of real-time breathing data recorded from the participant’s Zephyr BioHarness (https://www.zephyranywhere.com) and translated this data to control the loudness of the principal melodic line in the ambient music mix. An amplitude envelope, calculated using a square-root function normally used to generate stereo panning curves, was applied to the overall sound mixture in order to produce an undulating volume effect reminiscent of the time course of a normal inhale and exhale breathing cycle. A diagram of this envelope is included in the design illustration in Fig. 1. The difference in overall loudness between full “inhale” maxima and full “exhale” minima was 6 dB, representing a doubling of loudness during the course of a synthesized breath cycle. A sample of this music can be streamed at http://soundcloud.com/gracesounds.

We developed three intervention designs that differed only in the level of interaction between the musical composition engine and the participant’s real-time breathing data stream:

II-A Fixed Tempo (FT)

In the Fixed Tempo design, the synthesized breathing cycle was set at 6 breaths per minute (bpm), a rate shown to produce ideal levels of relaxation [34]. Therefore, this condition was equivalent to playing a pre-recorded piece of music designed to have relaxing qualities, but lacking individualization to each user’s breathing pattern.

II-B Personalized Tempo (PT)

We calculated each participant’s average baseline breathing rate at the beginning of the experiment session. During the Personalized Tempo block, the music was presented with amplitude modulations occurring at 75% of the calculated baseline breathing rate, in an effort to gently “nudge” a reduction in breathing rate. In any case of a participant having an abnormally high natural breathing rate, their Personalized Tempo was capped at 15 beats per minute.

II-C Personalized Envelope (PE)

In the most interactive design, the real-time breathing data stream fully governed the rate of synthesized “inhale” and “exhale,” producing a sympathetic music feedback which mirrored the exact time course of each participant’s unintentional inhales and exhales. While PT relies on a single number calculated during a participant’s Baseline session, PE uses one’s instantaneous breathing signal to generate each note.

III Methods

III-A Experimental Procedure

This study was pre-approved by the MIT Review Board. Our sample featured 19 participants (11 females, 8 males), of varying ages (19–55 years).

The experiments were conducted indoors in a sound-treated studio with no audible external noise. Participants were seated at a table in front of a laptop at a distance of approximately 2 meters from each of a pair of studio audio monitors (Genelec) placed at approximately 30°left and right of center. During data collection, participants were asked to keep still, breathe spontaneously, and fixate their vision on a computer-generated crosshair symbol. We also recorded their breathing waveform, and electrocardiogram (ECG) data using a Zephyr BioHarness (ZephyrTM Performance Systems) sampled at 17 Hz, and 250Hz, respectively. Bilateral electrodermal activity (EDA) at 4 Hz was recorded from right and left wrists using the E4 wristband (Empatica, Inc). See the system diagram in Fig. 1.

Prior to data collection, the wristbands and Zephyr Bioharness were placed on each participant, and they were asked to walk up and down three flights of stairs in order to properly prime the EDA, ECG, and respiration sensors. After the sensor priming, participants were outfitted with a 16-channel EasyCap EEG cap connected to a BrainVision VAmp EEG amplifier.

Participants completed four blocks of 40 forewarned reaction time trials; each block lasted an average of seven minutes due to random inter-trial intervals of between two and five seconds. For each trial, a warning stimulus was presented, followed by a fixed silent interval of 4.5 seconds. After this fixed interval, the imperative stimulus, an alarming short buzzing sound, was played. The participants were instructed to press a key on a computer keyboard as soon as they hear the imperative stimulus. This trial design is a classic stimulus presentation from the event-related potential (ERP) literature in EEG research, and is designed to elicit the contingent negative variation (CNV), a slow cortical potential whose amplitude increases with the focusing of attentional resources required for this task[35]. Since this task is demanding, it provides the opportunity to test the calming influence of our interventions on peripheral and cortical arousal.

In order to minimize muscle artifacts in EEG, participants were reminded not to blink or move their eyes unless absolutely necessary, to keep their eyes open as much as possible while still blinking naturally, to keep their eyes fixated on the displayed crosshair, and to not clench their muscles or move their right hand unnecessarily, even when they were trying to respond very quickly to the buzzer sound.

The first of the four blocks was presented silently, in order to collect baseline physiological measurements. In each of the subsequent three blocks, one of the interactive music interventions was introduced at random.

The Lab Streaming Layer [36] protocol provided real-time synchronization of stimulus presentation timestamps along with breathing, ECG, and EEG streams. EDA data were synchronized with the other streams post-hoc using the timestamps recorded on the E4.

All participants provided prior informed consent for the primary reaction time task with accompanying musical stimuli. Participants were told that the music was provided as entertainment to mitigate possible boredom during long blocks of the repetitive task, and that multiple physiological measurements would be taken in order to test the effects of focused attention on these bodily processes. This deceptive experiment design was necessary to ensure that participants would not attempt to consciously entrain their breathing to the music, or otherwise manipulate their breathing rate to achieve any particular result. After data collection was complete, each participant was informed of the real purpose of the experiment.

III-B Pre-processing Steps

The raw breathing waveform was measured as a time-series representation of the extent to which the participants’ breathing increased and decreased tension in the BioHarness chest strap. The breathing waveform data were filtered using a lowpass Butterworth filter with a cutoff frequency of 1 Hz, representing the fastest reasonable breathing rate. Local maxima of the breathing signal were calculated. We enforced the detected peaks to drop at least 2 $nu$s111$nu$ refers to non-unit, a relative pressure unit for raw breathing waveform. on either side before the signal attains a higher value. After detecting the peak locations, the inter-respiration intervals (IRI) were calculated in milliseconds. To better highlight the influence of interventions and lower the influence of personal baselines, we standardized measures for each participant. Specifically, the inter-respiration intervals were z-scored per participant using the following formula:

[TABLE]

Here, $z_{IRI}$ refers to the z-score of IRIs. Moreover, $\mu_{IRI}$ refers to the mean of the signal during a complete session of the participant, including Baseline, Fixed Tempo, Personalized Envelope, and Personalized Tempo conditions. Similarly, $\sigma_{IRI}$ refers to the standard deviation of the IRIs during the whole session. The mean of $z_{IRI}$ was calculated per block per participant as a proxy for the inverse of the relative breathing rate. The standard deviation of $z_{IRI}$ was calculated per block per participant as a proxy for variability of breathing.

Raw EDA was measured using the E4 sensors in $\mu S$. We applied a 6th order Butterworth low-pass filter (1 Hz cutoff frequency) to the EDA data. The filtered EDA was transformed into z-scores ($z_{EDA}$) similar to the method described in Equation 1. To better capture the relaxation response, we focused on the tonic skin conductance level which measures the smooth underlying slowly changing levels of EDA. Specifically, we measured the rate of change in the skin conductance level per experiment block per participant using the following formula:

[TABLE]

Here, $fs$ refers to the frequency of sampling which is 4Hz. Also, $n$ refers to the length of the EDA signal.

All EEG data processing was performed offline using the EEGLAB [37] toolbox for Matlab. Continuous EEG data were high-pass filtered at .05 Hz by subtracting a low-passed version of the signal (.05 Hz) from the original signal. Channels with a kurtosis greater than five were excluded from analysis, and the remaining channels were re-referenced to their average. The continuous EEG data were epoched around the first audio stimulus of each trial, from 1 second prior to the stimulus to 4 seconds post-stimulus, and the 500ms preceding the stimulus was used as the baseline. Individual epochs with absolute amplitudes exceeding 50uV were excluded from analysis. We calculated the mean amplitude in the Cz channel over three time windows representing the early (400-1400 ms), mid (1500-2600 ms), and late-stage (2600-3700 ms) CNV [38].

We used the Pan-Tompkins implementation of the QRS complex detector for ECG analyses [39]. The R-R intervals, i.e. inter-beat intervals (IBI), were consequently calculated. We used the Python HRV package [40] to calculate a range of time-based, frequency-based, and non-linear heart rate variability features from the IBIs. We also transformed the IBIs into z-scores similarly as in Equation (1) by standardizing IBIs within each session ($z_{IBI}$).

IV Results and Discussion

IV-A Comparing Interventions

In this section, we present our findings regarding physiological changes arising from ambient music conditions in comparison to baseline. We use box-plot [41], where the middle line represents median, the inner-box covers first to third quartiles, and the whiskers extend the inner-box boundaries 1.5 times inter-quartile range.

IV-A1 Breathing

Relaxation has multiple physiological indicators, including changes in respiration patterns. Deep and slow breathing both arise from and give rise to physiological, affective, and cognitive calm [42]. On the other hand, sustained attention and cognitive load have been shown to reduce respiratory variability [43]. Moreover, negative emotional states are shown to reduce correlated breathing variability [44] and sense of relief is associated with higher breathing variability [45]. In the aforementioned articles, tidal volume, instantaneous respiration rate, and minute ventilation and their coefficient of variation have been used to measure total respiratory variability. Additionally, autocorrelation at one breath lag has been used to quantify correlated respiratory variability. Breathing variability is also a predictor of respiratory health and has been associated with more successful separation of the patient from the ventilator [46, 47]. Given the rich literature on breathing and its relationship with affective states and wellbeing, there is a consensus that slower breathing rate and higher breathing variability are associated with a calmer state.

To quantify effects on calming, we thus focus on two metrics from the breathing signal: the z-score of inter-respiration intervals within each experiment block ($z_{IRI}$), as well as the variance of inter-respiration interval z-scores for each block ($var_{z_{IRI}}$). See §III-B for more information about how to calculate these features. $z_{IRI}$ is proportional to the inverse of relative breathing rate. Thus we expect to see higher $z_{IRI}$ in a more relaxed state. Additionally, $var_{z_{IRI}}$ is associated with breathing variability. Therefore, we expect to see higher $var_{z_{IRI}}$ in more positive and calming settings.

As shown in Fig. 2, our analyses revealed that there was a significant difference in z-score of inter-respiration intervals ($z_{IRI}$) across all the conditions. An ANOVA test was performed to evaluate differences in $z_{IRI}$ between Baseline, Fixed Tempo, Personalized Envelope, and Personalized Tempo designs: $F(3,72)=5.228,p=.003$. We conducted post-hoc pairwise comparisons using the independent t-test to compare each music design condition to the baseline: $t_{B-FT}=87.0,p_{B-FT}=.007;$ $t_{B-PE}=74.0,p_{B-PE}=.002;$ $t_{B-PT}=48.0,p_{B-PT}=.000;$ The Personalized Tempo design increases $z_{IRI}$ the most. The Personal Envelope design may influence breathing most strongly, but because of the variance between fast and slow breathers, it has a lesser effect overall. Moreover, simply having a fixed slow music tempo also reduces breathing (Fixed Tempo design), but has a lesser effect than the personalized designs.

Additionally, our analyses revealed that there was a significant difference in variability of inter-respiration interval z-scores ($var_{z_{IRI}}$) across all the conditions, as shown in Fig. 3. An ANOVA test was performed to evaluate differences in $var_{z_{IRI}}$ between Baseline, Fixed Tempo, Personalized Envelope, and Personalized Tempo designs: $F(3,72)=3.962,p=.011$. We conducted post-hoc pairwise comparisons using the independent t-test to compare each music design condition to the baseline: $t_{B-FT}=88.0,p_{B-FT}=.007;$ $t_{B-PE}=66.0,p_{B-PE}=.001;$ $t_{B-PT}=65.0,p_{B-PT}=.001;$ We observe that the music conditions resulted in a more variable breathing pattern which is associated with a more positive state. Moreover, the difference in breathing variability is more prominent in the personalized designs, suggesting that there is added value in bringing physiology-driven design to ambient music listening.

IV-A2 Electrodermal Activity (EDA)

EDA is traditionally characterized into two types, tonic and phasic activity. Tonic activity or skin conductance level shows the slowly-changing patterns of EDA. Lower levels of tonic activity are associated with more calming states. However, phasic activity or skin conductance response corresponds to rapid changes in EDA level in the form of peaks with a particular morphology: a quick rise and a slow decay. Higher skin conductance is usually associated with higher sympathetic stimulation and higher stress [48]. Since our experiment design presented stimuli with the goal of creating a relaxation response, we expect the tonic portion of the EDA to be more indicative of the effectiveness of our interactive music. As recommended by [48], we use z-scoring for standardization of raw EDA and use $slope_{z_{EDA}}$ as a metric of rate of change in skin conductance level. See §III-B for more information about feature calculation. In our study, participants were using their dominant hand for doing the primary reaction-time task and making mouse clicks continuously. This resulted in motion artifacts introduced to the right hand EDA signal. Thus, for this analysis, we focus on the non-dominant EDA, which has a long history of study in the skin conductance literature [48].

As shown in Fig. 4, our analyses revealed that there was a significant difference in rate of change in skin conductance level ($slope_{z_{EDA}}$) across all the conditions. An ANOVA test was performed to evaluate differences in $slope_{z_{EDA}}$ between Baseline, Fixed Tempo, Personalized Envelope, and Personalized Tempo designs: $F(3,68)=3.158,p=.030$222The left E4 sensor did not record any data for one participant. This resulted in 4 missing data-points, one per condition, for this analysis.. We conducted post-hoc pairwise comparisons using the independent t-test to compare each music condition to the baseline. We only observed significant differences across the Personalized Tempo design and the baseline333$t_{B-FT}=219.0,p_{B-FT}=.074;$ $t_{B-PE}=191.0,p_{B-PE}=.367;$ : $t_{B-PT}=228.0,p_{B-PT}=.038;$ This analysis shows that the Personalized Tempo design was specifically more influential in inducing a physiological state of calm and resulted in decreasing levels of tonic EDA.

IV-A3 Electroencephalogram (EEG)

We gave participants a reaction-time task that was chosen to elicit the contingent negative variation, a well-studied slow cortical potential that is known to index cortical arousal [35]. Past studies have shown that a decrease in EDA due to habituation to an auditory stimulus is accompanied by an increase in measured EEG power[49, 50]. Others have demonstrated that relaxation-inducing biofeedback causing a decrease in autonomic activity measured peripherally was also accompanied by an increase in the CNV amplitude [51]. As shown in Fig. 5, our analyses revealed that there was a significant difference between the amplitude of the late-stage CNV when compared between the baseline condition and all three design conditions. An ANOVA test was performed to evaluate differences in the late-stage CNV amplitude between the Baseline, Fixed Tempo, Personalized Envelope, and Personalized Tempo designs: $F(3,61)=3.180,p=.031$444Note that 8 datapoints had kurtosis greater than 5 (2 per each condition) and 3 were detected as outliers for having a z-score of more than 3 or less than -3 (1 per each intervention condition).. We conducted post-hoc pairwise comparisons using the independent t-test to compare each music design condition to the baseline. We only observed significant differences across the Personalized Tempo design and the baseline 555$t_{B-FT}=1.988,p_{B-FT}=.056;$ $t_{B-PE}=0.600,p_{B-PE}=.553;$ : $t_{B-PT}=2.556,p_{B-PT}=.016;$ The analysis of our breathing and EDA data shows that the Personalized Tempo design was specifically more influential in inducing a physiological state of calm. The accompanying result showing an influence on slow cortical potentials suggests that the decrease in peripherally measured autonomic activity caused an increase in cortical excitability, replicating findings of previous independent studies [49, 50].

IV-A4 Electrocardiogram (ECG)

We calculated the z-score of the inter-beat intervals ($z_{IBI}$). See §III-B for more information about this feature. $z_{IBI}$ is proportional to the inverse of relative heart rate; thus we expect to see a higher $z_{IBI}$ in a more relaxed state. Fig. 6, visualizes the difference between inter-beat interval z-scores ($z_{IBI}$) across all conditions. An ANOVA test was performed to evaluate differences in $z_{IBI}$ between Baseline, Fixed Tempo, Personalized Envelope, and Personalized Tempo designs: $F(3,68)=1.984,p=.125$666Experiment blocks with less than two minutes of valid IBIs (one data point per each condition) were excluded from this analysis.. We also report pairwise comparisons using an independent t-test that compare each music design condition to the baseline. We only observed a significant difference between Fixed Tempo and Baseline conditions777$t_{B-PE}=-1.159,p_{B-PE}=.255;$ $t_{B-PT}=-1.827,p_{B-PT}=.077;$ : $t_{B-FT}=-2.104,p_{B-FT}=.043;$

Though we calculated a comprehensive list of heart rate variability (HRV) features from ECG [40], we did not observe any significant differences between baseline and intervention conditions. This finding is to be expected, given that traditional HRV features may not best represent how the autonomic nervous system (ANS) is influencing heart function [52]. HRV is controlled by both the sympathetic and parasympathetic branches of the ANS. Traditional HRV measures are incapable of isolating the effects of these two branches, especially while breathing is changing. Particularly at low respiration rates, the parasympathetic activity shifts into lower frequencies and overlaps with the frequency interval that is traditionally associated with sympathetic activity [52]. Given that our ambient music design conditions have resulted in slower and more variable breathing, traditional HRV features are not equipped to distinguish between parasympathetic and sympathetic control of the heart [52]. See §IV-B for future work directions to try to mitigate this problem.

IV-B Limitations

In this paper, we solely focused on the left wrist EDA signal due to the motion artifacts introduced to the EDA captured from the right wrist. Recent findings show that a more holistic view of the EDA signal from multiple locations on the body could reveal further information about the affective and cognitive state of the user [53]. In the future, we would like to explore alternative signal processing techniques to overcome motion artifacts and study EDA asymmetry features.

We focused on traditional HRV measures for studying the influence of ANS on the heart. However, we learned that these features are not suitable over slow and variable breathing rates [52]. For future work, we would like to explore enhanced HRV features [52] that better distinguish sympathetic and parasympathetic indices of HRV in slow and variable breathing.

In addition, we did not inquire of participants after the experiment if they suspected that the real intention of the study was to manipulate their breathing pattern. In the future, these steps will be taken to ensure that analyzed data is only from participants unaware of the the music presentation intention. Future experiments are needed to test generalizability to real-world tasks and test if the influence of the music intervention extends to arousing effects using uptempo, arousing stimuli.

V Conclusion

This study engineered music in a systematic way to influence breathing in participants, despite that their focus was on a (different) cognitive task. The three music interactions differed with respect to the amount of customization to the participant’s own natural breathing rates. While previous designs have shown promise in encouraging entrainment of breathing patterns to external stimuli, this study was the first of its kind to specifically target unfocused entrainment using a deceptive experiment design to ensure participants did not consciously mimic the stimuli with their breath.

Our results revealed that the intricate design of musical stimuli did influence the participants’ breathing patterns. All intervention conditions resulted in higher relative inter-breath intervals, i.e. lower relative breathing rates, which are associated with a calmer state. Additionally, they influenced the relative variability of breathing which is also associated with a more positive and relaxed state. Importantly, personalizing the system based on the user’s natural breathing rate made the system significantly more influential to the user’s breathing pattern in addition to measures of their relaxation response, as seen in the downward shift in the tonic activity of EDA. Similarly, it resulted in greater cortical arousal as measured by the CNV, which has been shown to have an inverse association with peripheral arousal.

VI Acknowledgments

We thank Brain Vision LLC (Cary, NC, USA) for providing the EEG equipment and MIT Media Lab Consortium for supporting this research.