AU Dataset for Visuo-Haptic Object Recognition for Robots

TL;DR

This paper introduces a new multimodal dataset for visuo-haptic object recognition, containing visual, kinesthetic, and tactile data for 63 objects, to advance research in sensory integration for robots.

Contribution

It provides a publicly available, comprehensive multimodal dataset with detailed structure and methodology, addressing the scarcity of such datasets in the field.

Findings

Dataset includes multimodal data for 63 objects

Details on data collection and object properties

Facilitates research in sensory fusion for robotics

Abstract

Multimodal object recognition is still an emerging field. Thus, publicly available datasets are still rare and of small size. This dataset was developed to help fill this void and presents multimodal data for 63 objects with some visual and haptic ambiguity. The dataset contains visual, kinesthetic and tactile (audio/vibrations) data. To completely solve sensory ambiguity, sensory integration/fusion would be required. This report describes the creation and structure of the dataset. The first section explains the underlying approach used to capture the visual and haptic properties of the objects. The second section describes the technical aspects (experimental setup) needed for the collection of the data. The third section introduces the objects, while the final section describes the structure and content of the dataset.