Test Framework for a Virtual Competition Testbed

TL;DR

This paper presents a virtual competition testbed for service robotics, enabling safe, accessible, and realistic assessment of robot systems through an extendable virtual environment, especially useful during pandemic restrictions.

Contribution

It introduces a virtual scoring environment for service robotics competitions, facilitating system evaluation without physical access and promoting broader participation.

Findings

Virtual environment enables realistic task assessment

Supports increased access and collaboration

Provides a foundation for future virtual competitions

Abstract

Virtual environments have been utilised in robotics research as a tool to assess systems before deploying them in the field. The COVID-19 pandemic has brought about additional motivation for the development of virtual benchmarks in order to aid in safe and productive development. In-person robotics competitions have also halted, thus limiting the scope of opportunities for students and researchers. We implemented the structure of a service robotics competition into an extendable and adaptable virtual scoring environment. The competition challenges the state of the art in home service robotics by presenting realistic household tasks for robots to complete. The virtual environment provides a foundation for competition teams to assess their systems when accessing the physical environment is not possible. We believe that utilising virtual environments as a means of assessment will lead to other benefits such as increased access and generalisation.