Mixed Reality Interaction Techniques

TL;DR

This chapter reviews diverse interaction techniques for mixed reality environments, covering input/output modalities, multisensory integration, multi-display interaction, and virtual agents, highlighting current approaches and future directions.

Contribution

It provides a comprehensive overview of mixed reality interaction techniques, integrating various modalities and multi-display interactions, and discusses interaction with virtual agents.

Findings

Various input/output modalities are used in MR.

Multisensory and multimodal interactions enhance user experience.

Interaction with virtual agents is an emerging area.

Abstract

This chapter gives an overview of interaction techniques for mixed reality including augmented and virtual reality (AR/VR). Various modalities for input and output are discussed. Specifically, techniques for tangible and surface-based interaction, gesture-based, pen-based, gaze-based, keyboard and mouse-based, as well as haptic interaction are discussed. Furthermore, the combination of multiple modalities in multisensory and multimodal interaction, as well as interaction using multiple physical or virtual displays, are presented. Finally, interaction with intelligent virtual agents is considered.