CREATE: Multimodal Dataset for Unsupervised Learning, Generative Modeling and Prediction of Sensory Data from a Mobile Robot in Indoor Environments

TL;DR

The CREATE dataset provides extensive multimodal sensory data from a mobile robot, enabling research in unsupervised learning, generative modeling, and prediction of environmental and internal states in indoor settings.

Contribution

This paper introduces a comprehensive multimodal dataset from a mobile robot, facilitating new research in multimodal representation learning and sensor data prediction.

Findings

Dataset includes 14 hours of multimodal recordings

Supports multimodal unsupervised object learning and prediction

Enables study of sensor dependencies and robot-environment interactions

Abstract

The CREATE database is composed of 14 hours of multimodal recordings from a mobile robotic platform based on the iRobot Create. The various sensors cover vision, audition, motors and proprioception. The dataset has been designed in the context of a mobile robot that can learn multimodal representations of its environment, thanks to its ability to navigate the environment. This ability can also be used to learn the dependencies and relationships between the different modalities of the robot (e.g. vision, audition), as they reflect both the external environment and the internal state of the robot. The provided multimodal dataset is expected to have multiple usages, such as multimodal unsupervised object learning, multimodal prediction and egomotion/causality detection.