Metaversal Learning Environments: Measuring, predicting and improving interpersonal effectiveness

TL;DR

This paper introduces a novel AI and VR-based framework in the Metaverse to measure, assess, and improve interpersonal effectiveness through avatar interactions, demonstrating significant behavioral improvements and predictive capabilities.

Contribution

The work presents a new immersive architecture combining AI and VR for measuring and enhancing interpersonal skills, with a machine learning approach to predict effectiveness.

Findings

Individuals with interpersonal deficits improve after multiple avatar interactions

Participants interact naturally and exhibit real-world behavioral traits

Machine learning can predict interpersonal effectiveness from audio and video data

Abstract

Experiential learning has been known to be an engaging and effective modality for personal and professional development. The Metaverse provides ample opportunities for the creation of environments in which such experiential learning can occur. In this work, we introduce a novel architecture that combines Artificial intelligence and Virtual Reality to create a highly immersive and efficient learning experience using avatars. The framework allows us to measure the interpersonal effectiveness of an individual interacting with the avatar. We first present a small pilot study and its results which were used to enhance the framework. We then present a larger study using the enhanced framework to measure, assess, and predict the interpersonal effectiveness of individuals interacting with an avatar. Results reveal that individuals with deficits in their interpersonal effectiveness show a significant improvement in performance after multiple interactions with an avatar. The results also reveal that individuals interact naturally with avatars within this framework, and exhibit similar behavioral traits as they would in the real world. We use this as a basis to analyze the underlying audio and video data streams of individuals during these interactions. Finally, we extract relevant features from these data and present a machine-learning based approach to predict interpersonal effectiveness during human-avatar conversation. We conclude by discussing the implications of these findings to build beneficial applications for the real world.