Sonic Interactions in Virtual Environments: the Egocentric Audio Perspective of the Digital Twin

TL;DR

This paper proposes a theoretical framework for sonic interactions in virtual environments, emphasizing the egocentric auditory digital twin to enhance immersive, participatory experiences through dynamic intra-actions between humans and technology.

Contribution

It introduces the concept of an egocentric auditory digital twin as a new theoretical perspective for understanding sonic interactions in virtual environments.

Findings

The digital twin fosters intra-actions between humans and technology.

Egocentric perspective enhances perceptual understanding in VEs.

The framework supports dynamic, coherent immersive experiences.

Abstract

The relationships between the listener, physical world and virtual environment (VE) should not only inspire the design of natural multimodal interfaces but should be discovered to make sense of the mediating action of VR technologies. This chapter aims to transform an archipelago of studies related to sonic interactions in virtual environments (SIVE) into a research field equipped with a first theoretical framework with an inclusive vision of the challenges to come: the egocentric perspective of the auditory digital twin. In a VE with immersive audio technologies implemented, the role of VR simulations must be enacted by a participatory exploration of sense-making in a network of human and non-human agents, called actors. The guardian of such locus of agency is the auditory digital twin that fosters intra-actions between humans and technology, dynamically and fluidly redefining all those configurations that are crucial for an immersive and coherent experience. The idea of entanglement theory is here mainly declined in an egocentric-spatial perspective related to emerging knowledge of the listener's perceptual capabilities. This is an actively transformative relation with the digital twin potentials to create movement, transparency, and provocative activities in VEs. The chapter contains an original theoretical perspective complemented by several bibliographical references and links to the other book chapters that have contributed significantly to the proposal presented here.