# COVERED, CollabOratiVE Robot Environment Dataset for 3D Semantic segmentation

## TL;DR

This paper introduces COVERED, a new dataset for 3D semantic segmentation in collaborative robot environments, and demonstrates real-time semantic segmentation with high accuracy using deep learning and multi-LiDAR systems.

## Contribution

The work provides the first dedicated dataset for 3D semantic segmentation in collaborative robot workspaces and benchmarks state-of-the-art algorithms on this dataset.

## Key findings

- Achieved over 96% point-wise accuracy and 92% mIOU in real-time segmentation.
- Demonstrated 20Hz processing throughput with high accuracy.
- Provided a baseline for future research in semantic understanding for HRC.

## Abstract

Safe human-robot collaboration (HRC) has recently gained a lot of interest with the emerging Industry 5.0 paradigm. Conventional robots are being replaced with more intelligent and flexible collaborative robots (cobots). Safe and efficient collaboration between cobots and humans largely relies on the cobot's comprehensive semantic understanding of the dynamic surrounding of industrial environments. Despite the importance of semantic understanding for such applications, 3D semantic segmentation of collaborative robot workspaces lacks sufficient research and dedicated datasets. The performance limitation caused by insufficient datasets is called 'data hunger' problem. To overcome this current limitation, this work develops a new dataset specifically designed for this use case, named "COVERED", which includes point-wise annotated point clouds of a robotic cell. Lastly, we also provide a benchmark of current state-of-the-art (SOTA) algorithm performance on the dataset and demonstrate a real-time semantic segmentation of a collaborative robot workspace using a multi-LiDAR system. The promising results from using the trained Deep Networks on a real-time dynamically changing situation shows that we are on the right track. Our perception pipeline achieves 20Hz throughput with a prediction point accuracy of $>$96\% and $>$92\% mean intersection over union (mIOU) while maintaining an 8Hz throughput.