Multi-person Spatial Interaction in a Large Immersive Display Using Smartphones as Touchpads

TL;DR

This paper introduces a multi-user interaction system for large immersive displays that combines smartphone touch input, spatial tracking, and web interfaces to enable natural, simultaneous multi-person interactions.

Contribution

It presents a novel multi-user interaction framework integrating smartphones, Kinect-based spatial tracking, and web interfaces for immersive display environments.

Findings

Users successfully performed image selection and placement tasks.

The system supports dynamic, multi-person interactions in large display spaces.

The framework enables natural multimodal interaction in immersive environments.

Abstract

In this paper, we present a multi-user interaction interface for a large immersive space that supports simultaneous screen interactions by combining (1) user input via personal smartphones and Bluetooth microphones, (2) spatial tracking via an overhead array of Kinect sensors, and (3) WebSocket interfaces to a webpage running on the large screen. Users are automatically, dynamically assigned personal and shared screen sub-spaces based on their tracked location with respect to the screen, and use a webpage on their personal smartphone for touchpad-type input. We report user experiments using our interaction framework that involve image selection and placement tasks, with the ultimate goal of realizing display-wall environments as viable, interactive workspaces with natural multimodal interfaces.