Mobile Delivery Robots: Mixed Reality-Based Simulation Relying on ROS and Unity 3D

TL;DR

This paper introduces a mixed reality simulation method combining ROS and Unity 3D to enhance the realism and reliability of autonomous delivery robot testing in urban environments.

Contribution

It presents a novel MR simulation approach that synchronizes real and virtual robots using sensor data, improving testing accuracy for delivery robots.

Findings

Verified feasibility and stability of MR simulation with Unity for ROS robots

Demonstrated improved perception and interaction between real and virtual robots

Enhanced simulation realism for urban delivery scenarios

Abstract

In the context of Intelligent Transportation Systems and the delivery of goods, new technology approaches need to be developed in order to cope with certain challenges that last mile delivery entails, such as navigation in an urban environment. Autonomous delivery robots can help overcome these challenges. We propose a method for performing mixed reality (MR) simulation with ROS-based robots using Unity, which synchronizes the real and virtual environment, and simultaneously uses the sensor information of the real robots to locate themselves and project them into the virtual environment, so that they can use their virtual doppelganger to perceive the virtual world. Using this method, real and virtual robots can perceive each other and the environment in which the other party is located, thereby enabling the exchange of information between virtual and real objects. Through this approach a more realistic and reliable simulation can be obtained. Results of the demonstrated use-cases verified the feasibility and efficiency as well as the stability of implementing MR using Unity for ROS-based robots.