Breaking the mould of Social Mixed Reality - State-of-the-Art and Glossary

TL;DR

This paper reviews the current state of Social Mixed Reality, emphasizing the need for human-centric design, multi-modal data integration, and ethical considerations to enable authentic social interactions with virtual agents.

Contribution

It provides a comprehensive glossary of key concepts and challenges in Social MR, advocating for human-centric, ethical, and inclusive technological advancements.

Findings

Highlighting the gap in authentic embodiment and social interaction in MR

Proposing a framework for multi-modal data and multi-agent interaction

Emphasizing ethical and inclusive design principles in Social MR

Abstract

This article explores a critical gap in Mixed Reality (MR) technology: while advances have been made, MR still struggles to authentically replicate human embodiment and socio-motor interaction. For MR to enable truly meaningful social experiences, it needs to incorporate multi-modal data streams and multi-agent interaction capabilities. To address this challenge, we present a comprehensive glossary covering key topics such as Virtual Characters and Autonomisation, Responsible AI, Ethics by Design, and the Scientific Challenges of Social MR within Neuroscience, Embodiment, and Technology. Our aim is to drive the transformative evolution of MR technologies that prioritize human-centric innovation, fostering richer digital connections. We advocate for MR systems that enhance social interaction and collaboration between humans and virtual autonomous agents, ensuring inclusivity, ethical design and psychological safety in the process.