Immersive Rover Control and Obstacle Detection based on Extended Reality and Artificial Intelligence

TL;DR

This paper introduces an innovative system combining XR and AI for lunar rover teleoperation, enabling autonomous obstacle detection and immersive 3D environment recreation to enhance remote lunar exploration.

Contribution

It presents a novel integrated XR and AI system for lunar rover control that improves obstacle detection and immersive visualization over traditional methods.

Findings

System successfully detects rocks autonomously

Creates immersive 3D virtual environments

Outperforms traditional 2D teleoperation approaches

Abstract

Lunar exploration has become a key focus, driving scientific and technological advances. Ongoing missions are deploying rovers to the surface of the Moon, targeting the far side and south pole. However, these terrains pose challenges, emphasizing the need for precise obstacles and resource detection to avoid mission risks. This work proposes a novel system that integrates eXtended Reality (XR) and Artificial Intelligence (AI) to teleoperate lunar rovers. It is capable of autonomously detecting rocks and recreating an immersive 3D virtual environment of the location of the robot. This system has been validated in a lunar laboratory to observe its advantages over traditional 2D-based teleoperation approaches