Chemistry3D: Robotic Interaction Benchmark for Chemistry Experiments

TL;DR

Chemistry3D is a comprehensive simulation toolkit that enables robotic chemical experiments with real-time visualization and supports RL tasks, object detection, and sim-to-real validation, advancing robotic manipulation in chemistry.

Contribution

It introduces Chemistry3D, a novel platform integrating chemical and robotic knowledge for simulation, visualization, and manipulation of chemical experiments in robotics.

Findings

Successful simulation of chemical reactions with real-time visualization

Effective RL tasks and object detection in chemical scenarios

Validated simulation performance through Sim2Real experiments

Abstract

The advent of simulation engines has revolutionized learning and operational efficiency for robots, offering cost-effective and swift pipelines. However, the lack of a universal simulation platform tailored for chemical scenarios impedes progress in robotic manipulation and visualization of reaction processes. Addressing this void, we present Chemistry3D, an innovative toolkit that integrates extensive chemical and robotic knowledge. Chemistry3D not only enables robots to perform chemical experiments but also provides real-time visualization of temperature, color, and pH changes during reactions. Built on the NVIDIA Omniverse platform, Chemistry3D offers interfaces for robot operation, visual inspection, and liquid flow control, facilitating the simulation of special objects such as liquids and transparent entities. Leveraging this toolkit, we have devised RL tasks, object detection, and robot operation scenarios. Additionally, to discern disparities between the rendering engine and the real world, we conducted transparent object detection experiments using Sim2Real, validating the toolkit's exceptional simulation performance. The source code is available at https://github.com/huangyan28/Chemistry3D, and a related tutorial can be found at https://www.omni-chemistry.com.