Perspective-Aware AI in Extended Reality
Daniel Platnick, Matti Gruener, Marjan Alirezaie, Kent Larson, Dava J. Newman, Hossein Rahnama

TL;DR
This paper introduces PAiR, a framework that integrates perspective-aware AI with XR to create more adaptive, interpretable, and user-specific immersive experiences by modeling user identity and cognitive states.
Contribution
The paper presents a novel framework, PAiR, that combines perspective-aware AI with XR using identity models learned from multimodal data, enabling context-aware and interpretable experiences.
Findings
Demonstrated PAiR's architecture and modules in Unity-based OpenDome.
Implemented two proof-of-concept scenarios showing practical utility.
Showed potential for improved human-AI interaction in XR environments.
Abstract
AI-enhanced Extended Reality (XR) aims to deliver adaptive, immersive experiences-yet current systems fall short due to shallow user modeling and limited cognitive context. We introduce Perspective-Aware AI in Extended Reality (PAiR), a foundational framework for integrating Perspective-Aware AI (PAi) with XR to enable interpretable, context-aware experiences grounded in user identity. PAi is built on Chronicles: reasoning-ready identity models learned from multimodal digital footprints that capture users' cognitive and experiential evolution. PAiR employs these models in a closed-loop system linking dynamic user states with immersive environments. We present PAiR's architecture, detailing its modules and system flow, and demonstrate its utility through two proof-of-concept scenarios implemented in the Unity-based OpenDome engine. PAiR opens a new direction for human-AI interaction by…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsImage Retrieval and Classification Techniques · 3D Surveying and Cultural Heritage · Advanced Image and Video Retrieval Techniques
