What you see is (not) what you get: A VR Framework for Correcting Robot Errors

TL;DR

This paper introduces a VR framework that enables users to identify, correct, and modify robot errors in real-time, improving robustness in robot perception and planning through interactive virtual reality corrections.

Contribution

The paper presents a novel VR framework allowing users to detect and correct robot failures, integrating virtual corrections with real-world robot behaviors.

Findings

User study demonstrates improved error correction efficiency

Framework enhances robustness of robot perception and planning

Open-source materials available for replication

Abstract

Many solutions tailored for intuitive visualization or teleoperation of virtual, augmented and mixed (VAM) reality systems are not robust to robot failures, such as the inability to detect and recognize objects in the environment or planning unsafe trajectories. In this paper, we present a novel virtual reality (VR) framework where users can (i) recognize when the robot has failed to detect a real-world object, (ii) correct the error in VR, (iii) modify proposed object trajectories and, (iv) implement behaviors on a real-world robot. Finally, we propose a user study aimed at testing the efficacy of our framework. Project materials can be found in the OSF repository.