# Simitate: A Hybrid Imitation Learning Benchmark

## TL;DR

Simitate is a comprehensive benchmarking suite with a new dataset and evaluation tools designed to advance imitation learning research in realistic, simulated environments.

## Contribution

It introduces a large, realistic dataset with synchronized RGB, depth, and ground truth data, along with a benchmarking suite for standardized evaluation of imitation learning methods.

## Key findings

- Provides a new dataset with 1938 human activity sequences
- Includes integrated simulation environment with ground truth data
- Offers evaluation metrics for imitation quality

## Abstract

We present Simitate --- a hybrid benchmarking suite targeting the evaluation of approaches for imitation learning. A dataset containing 1938 sequences where humans perform daily activities in a realistic environment is presented. The dataset is strongly coupled with an integration into a simulator. RGB and depth streams with a resolution of 960$\mathbb{\times}$540 at 30Hz and accurate ground truth poses for the demonstrator's hand, as well as the object in 6 DOF at 120Hz are provided. Along with our dataset we provide the 3D model of the used environment, labeled object images and pre-trained models. A benchmarking suite that aims at fostering comparability and reproducibility supports the development of imitation learning approaches. Further, we propose and integrate evaluation metrics on assessing the quality of effect and trajectory of the imitation performed in simulation. Simitate is available on our project website: \url{https://agas.uni-koblenz.de/data/simitate/}.