Seamless and Efficient Interactions within a Mixed-Dimensional Information Space

TL;DR

This paper explores the design of seamless interaction techniques within a mixed-dimensional information space, integrating 2D and 3D entities for more efficient user experiences across digital and physical environments.

Contribution

It introduces new interactive systems and techniques to enable seamless transitions and interactions within a mixed-dimensional information space involving 2D and 3D entities.

Findings

MemoVis uses AI to assist in 3D design feedback.

PaperToPlace transforms paper instructions into MR experiences.

VRContour enhances contour delineation workflows in VR.

Abstract

Mediated by today's visual displays, information space allows users to discover, access and interact with a wide range of digital and physical information. The information presented in this space may be digital, physical or a blend of both, and appear across different dimensions - such as texts, images, 3D content and physical objects embedded within real-world environment. Navigating within the information space often involves interacting with mixed-dimensional entities, visually represented in both 2D and 3D. At times, interactions also involve transitioning among entities represented in different dimensions. We introduce the concept of mixed-dimensional information space, encompassing entities represented in both 2D and 3D. Interactions within the mixed-dimensional information space should be seamless and efficient: users should be able to focus on their primary tasks without being distracted by interactions with or transitions between entities. While incorporating 3D representations into the mixed-dimensional information space offers intuitive and immersive ways to interact with complex information, it is important to address potential seams and inefficiencies that arise while interacting with both 2D and 3D entities. This dissertation introduces new interactive techniques and systems to realize seamless and efficient interactions within the mixed-dimensional information space. This dissertation introduces three interactive systems: MemoVis which aims to use emergent generative AI to help users create reference images for 3D design feedback; PaperToPlace which demonstrates how paper-based instruction documents can be transformed and spatialized into a context-aware MR experience; and VRContour which explores how contour delineation workflow can be brought into VR.