If Turing played piano with an artificial partner

TL;DR

This study explores whether neural network-based artificial partners can create a convincing social piano-playing experience, finding that simplicity and interaction are key factors over generative complexity.

Contribution

It demonstrates that a variational autoencoder trained on musical scores can serve as an artificial partner in social piano playing, emphasizing interaction over complexity.

Findings

Artificial partners are rated lower than humans but show promise.

Simpler artificial partners can match human ratings on some measures.

Interactive design is more crucial than generative sophistication.

Abstract

Music is an inherently social activity that allows people to share experiences and feel connected with one another. There has been little progress in designing artificial partners exhibiting a similar social experience as playing with another person. Neural network architectures that implement generative models, such as large language models, are suited for producing musical scores. Playing music socially, however, involves more than playing a score; it must complement the other musicians' ideas and keep time correctly. We addressed the question of whether a convincing social experience is made possible by a generative model trained to produce musical scores, not necessarily optimized for synchronization and continuation. The network, a variational autoencoder trained on a large corpus of digital scores, was adapted for a timed call-and-response task with a human partner. Participants played piano with a human or artificial partner-in various configurations-and rated the performance quality and first-person experience of self-other integration. Overall, the artificial partners held promise but were rated lower than human partners. The artificial partner with simplest design and highest similarity parameter was not rated differently from the human partners on some measures, suggesting that interactive rather than generative sophistication is important in enabling social AI.