Virtual Reality for Robots

TL;DR

This paper explores applying Virtual Reality to robots, enabling them to experience blended real and virtual sensor inputs for improved testing, security, and learning, through novel methods and a mathematical framework.

Contribution

It introduces two new VR application methods for robots, black box and white box, and discusses their potential uses and implications.

Findings

Enhanced testing and verification capabilities

Potential for studying spoofing and anti-spoofing

Sample generation for machine learning

Abstract

This paper applies the principles of Virtual Reality (VR) to robots, rather than living organisms. A simulator, of either physical states or information states, renders outputs to custom displays that fool the robot's sensors. This enables a robot to experience a combination of real and virtual sensor inputs, combining the efficiency of simulation and the benefits of real world sensor inputs. Thus, the robot can be taken through targeted experiences that are more realistic than pure simulation, yet more feasible and controllable than pure real-world experiences. We define two distinctive methods for applying VR to robots, namely black box and white box; based on these methods we identify potential applications, such as testing and verification procedures that are better than simulation, the study of spoofing attacks and anti-spoofing techniques, and sample generation for machine learning. A general mathematical framework is presented, along with a simple experiment, detailed examples, and discussion of the implications.