Goal-oriented Semantic Communications for Metaverse Construction via Generative AI and Optimal Transport

TL;DR

This paper introduces a goal-oriented semantic communication framework for the metaverse that leverages generative AI and optimal transport to significantly reduce latency and improve content accuracy during wireless transmission.

Contribution

The paper proposes a novel GSC framework with HgNet encoder and OT-enabled denoiser, enhancing real-time metaverse updates over wireless channels.

Findings

92.6% reduction in construction latency

45.6% improvement in object status accuracy

44.7% better viewing experience

Abstract

The emergence of the metaverse has boosted productivity and creativity, driving real-time updates and personalized content, which will substantially increase data traffic. However, current bit-oriented communication networks struggle to manage this high volume of dynamic information, restricting metaverse applications interactivity. To address this research gap, we propose a goal-oriented semantic communication (GSC) framework for metaverse. Building on an existing metaverse wireless construction task, our proposed GSC framework includes an hourglass network-based (HgNet) encoder to extract semantic information of objects in the metaverse; and a semantic decoder that uses this extracted information to reconstruct the metaverse content after wireless transmission, enabling efficient communication and real-time object behaviour updates to the scenery for metaverse construction task. To overcome the wireless channel noise at the receiver, we design an optimal transport (OT)-enabled semantic denoiser, which enhances the accuracy of metaverse scenery through wireless communication. Experimental results show that compared to the conventional metaverse construction, our proposed GSC framework significantly reduces wireless metaverse construction latency by 92.6\%, while improving metaverse object status accuracy and viewing experience by 45.6\% and 44.7\%, respectively.