All Reality: Virtual, Augmented, Mixed (X), Mediated (X,Y), and Multimediated Reality

TL;DR

This paper introduces a comprehensive taxonomy of realities including virtual, augmented, mixed, mediated, and new phenomenological realities, proposing a multidimensional, multisensory framework called Multimediated Reality that extends human perception.

Contribution

It provides a new taxonomy of realities and introduces Multimediated Reality, expanding the concept to include additional senses and phenomenological experiences beyond synthetic realities.

Findings

Proposes a multidimensional taxonomy of realities.

Introduces Multimediated Reality with multisensory and multiscale features.

Explores the inclusion of natural phenomena and additional senses in mediated experiences.

Abstract

The contributions of this paper are: (1) a taxonomy of the "Realities" (Virtual, Augmented, Mixed, Mediated, etc.), and (2) some new kinds of "reality" that come from nature itself, i.e. that expand our notion beyond synthetic realities to include also phenomenological realities. VR (Virtual Reality) replaces the real world with a simulated experience (virtual world). AR (Augmented Reality) allows a virtual world to be experienced while also experiencing the real world at the same time. Mixed Reality provides blends that interpolate between real and virtual worlds in various proportions, along a "Virtuality" axis, and extrapolate to an "X-axis". Mediated Reality goes a step further by mixing/blending and also modifying reality. This modifying of reality introduces a second axis. Mediated Reality is useful as a seeing aid (e.g. modifying reality to make it easier to understand), and for psychology experiments like Stratton's 1896 upside-down eyeglasses experiment. We propose Multimediated Reality as a multidimensional multisensory mediated reality that includes not just interactive multimedia-based reality for our five senses, but also includes additional senses (like sensory sonar, sensory radar, etc.), as well as our human actions/actuators. These extra senses are mapped to our human senses using synthetic synesthesia. This allows us to directly experience real (but otherwise invisible) phenomena, such as wave propagation and wave interference patterns, so that we can see radio waves and sound waves and how they interact with objects and each other. Multimediated reality is multidimensional, multimodal, multisensory, and multiscale. It is also multidisciplinary, in that we must consider not just the user, but also how the technology affects others, e.g. how its physical appearance affects social situations.