A Monocular SLAM-based Multi-User Positioning System with Image Occlusion in Augmented Reality

TL;DR

This paper presents a multi-user AR positioning system using monocular SLAM, virtual objects for synchronization, and deep learning for occlusion handling, enabling consistent multi-user experiences in AR environments.

Contribution

It introduces a novel multi-user localization framework based on ORB-SLAM2 integrated with Unity 3D, incorporating deep learning for occlusion management in AR.

Findings

Achieved accurate multi-user positioning with virtual object synchronization.

Implemented depth estimation from monocular images to handle occlusion.

Demonstrated natural virtual object rendering in multi-user AR scenes.

Abstract

In recent years, with the rapid development of augmented reality (AR) technology, there is an increasing demand for multi-user collaborative experiences. Unlike for single-user experiences, ensuring the spatial localization of every user and maintaining synchronization and consistency of positioning and orientation across multiple users is a significant challenge. In this paper, we propose a multi-user localization system based on ORB-SLAM2 using monocular RGB images as a development platform based on the Unity 3D game engine. This system not only performs user localization but also places a common virtual object on a planar surface (such as table) in the environment so that every user holds a proper perspective view of the object. These generated virtual objects serve as reference points for multi-user position synchronization. The positioning information is passed among every user's AR devices via a central server, based on which the relative position and movement of other users in the space of a specific user are presented via virtual avatars all with respect to these virtual objects. In addition, we use deep learning techniques to estimate the depth map of an image from a single RGB image to solve occlusion problems in AR applications, making virtual objects appear more natural in AR scenes.