# Enhancing human spatial awareness through augmented reality technologies

## TL;DR

This paper explores how augmented reality and deep learning can improve human spatial awareness in challenging environments like underwater and disaster zones.

## Contribution

The paper introduces an AR-enabled system that uses deep learning for 3D point cloud reconstruction and completion in limited perception environments.

## Key findings

- The system successfully recovers and augments unseen spatial information using 3D deep learning methods.
- Segmentation accuracy and reconstruction completeness were evaluated in synthetic and underwater trials.
- Challenges include data scarcity, noise, and the need for better metrics and labeled sonar datasets.

## Abstract

Augmented reality (AR) has emerged as a powerful tool for enhancing human spatial awareness by overlaying digital information onto the physical world. This paper presents a review of the methodologies that enable AR-based spatial perception, with a focus on challenging environments such as underwater and disaster scenarios. We review state-of-the-art deep learning approaches for 3D data interpretation and completion, including voxel-based, point-based, and view-based methods. As part of this review, we implement an AR-enabled spatial awareness system, where the investigated deep learning solutions can be tested directly. In our approach, a robotic arm with an ultrasound sensor performs 2D scans underwater, from which a 3D point cloud of the scene is reconstructed. Using the reviewed deep learning networks, the point cloud is segmented in order to identify objects of interest, and point cloud completion is performed to infer missing structure. We report experimental results from synthetic data and underwater scanning trials, demonstrating that the system can recover and augment unseen spatial information for the user. We discuss the outcomes, including segmentation accuracy and completeness of reconstructions, as well as challenges such as data scarcity, noise, and real-time constraints. The paper concludes that, when combined with robust sensing and 3D deep learning techniques, AR enhances human spatial awareness in environments where direct perception is limited. The need for more adequate metrics to describe point clouds and for more labeled sonar datasets is discussed.