Inferring Intentions to Speak Using Accelerometer Data In-the-Wild

TL;DR

This study explores using accelerometer data from smart badges to infer people's intentions to speak in real-world social settings, revealing limited success and the need for additional data modalities.

Contribution

It introduces a machine learning approach to infer speaking intentions from accelerometer data collected in natural environments, highlighting the challenges and limitations.

Findings

Accelerometer data contains some information about speaking intentions.

Posture shifts correlate with intentions but are not definitive indicators.

Current data modality is insufficient for reliable intention inference.

Abstract

Humans have good natural intuition to recognize when another person has something to say. It would be interesting if an AI can also recognize intentions to speak. Especially in scenarios when an AI is guiding a group discussion, this can be a useful skill. This work studies the inference of successful and unsuccessful intentions to speak from accelerometer data. This is chosen because it is privacy-preserving and feasible for in-the-wild settings since it can be placed in a smart badge. Data from a real-life social networking event is used to train a machine-learning model that aims to infer intentions to speak. A subset of unsuccessful intention-to-speak cases in the data is annotated. The model is trained on the successful intentions to speak and evaluated on both the successful and unsuccessful cases. In conclusion, there is useful information in accelerometer data, but not enough to reliably capture intentions to speak. For example, posture shifts are correlated with intentions to speak, but people also often shift posture without having an intention to speak, or have an intention to speak without shifting their posture. More modalities are likely needed to reliably infer intentions to speak.