Task-oriented and Semantics-aware Communications for Augmented Reality

TL;DR

This paper introduces TSAR, a novel semantics-aware communication framework for AR that significantly improves transmission efficiency and effectiveness by reducing latency and enhancing data quality.

Contribution

The paper proposes a new task-oriented, semantics-aware communication framework for AR, integrating deep learning-based semantics extraction and task-specific knowledge, outperforming traditional methods.

Findings

Reduces AR transmission latency by 95.6%.

Improves geometry and color communication effectiveness by up to 82.4% and 20.4%.

Demonstrates significant performance gains over traditional point cloud communication.

Abstract

Upon the advent of the emerging metaverse and its related applications in Augmented Reality (AR), the current bit-oriented network struggles to support real-time changes for the vast amount of associated information, creating a significant bottleneck in its development. To address the above problem, we present a novel task-oriented and semantics-aware communication framework for augmented reality (TSAR) to enhance communication efficiency and effectiveness significantly. We first present an analysis of the traditional wireless AR point cloud communication framework, followed by a detailed summary of our proposed semantic information extraction within the end-to-end communication. Then, we detail the components of the TSAR framework, incorporating semantics extraction with deep learning, task-oriented base knowledge selection, and avatar pose recovery. Through rigorous experimentation, we demonstrate that our proposed TSAR framework considerably outperforms traditional point cloud communication framework, reducing wireless AR application transmission latency by 95.6% and improving communication effectiveness in geometry and color aspects by up to 82.4% and 20.4%, respectively.