HoloBots: Augmenting Holographic Telepresence with Mobile Robots for Tangible Remote Collaboration in Mixed Reality

TL;DR

HoloBots combines holographic telepresence with synchronized mobile robots to enable remote users to physically interact with local environments, enhancing shared presence and tangible collaboration in mixed reality settings.

Contribution

This work introduces HoloBots, a novel system integrating holographic telepresence with mobile robots for physical interaction in remote collaboration.

Findings

Significantly increased co-presence and shared experience.

Effective synchronization of holograms with mobile robots.

Positive user feedback on physical engagement capabilities.

Abstract

This paper introduces HoloBots, a mixed reality remote collaboration system that augments holographic telepresence with synchronized mobile robots. Beyond existing mixed reality telepresence, HoloBots lets remote users not only be visually and spatially present, but also physically engage with local users and their environment. HoloBots allows the users to touch, grasp, manipulate, and interact with the remote physical environment as if they were co-located in the same shared space. We achieve this by synchronizing holographic user motion (Hololens 2 and Azure Kinect) with tabletop mobile robots (Sony Toio). Beyond the existing physical telepresence, HoloBots contributes to an exploration of broader design space, such as object actuation, virtual hand physicalization, world-in-miniature exploration, shared tangible interfaces, embodied guidance, and haptic communication. We evaluate our system with twelve participants by comparing it with hologram-only and robot-only conditions. Both quantitative and qualitative results confirm that our system significantly enhances the level of co-presence and shared experience, compared to the other conditions.