Predicting and Attending to Damaging Collisions for Placing Everyday Objects in Photo-Realistic Simulations

TL;DR

This paper introduces PonNet, a novel deep learning model with attention mechanisms that predicts damaging collisions in object placement tasks for domestic robots, supported by a new dataset of realistic images.

Contribution

The paper presents PonNet, an attention-based neural network for collision prediction, and provides a new dataset of photo-realistic images for training and evaluation.

Findings

PonNet outperforms baseline methods in collision prediction accuracy.

The dataset of 12,000 images enables robust training of placement risk models.

Visualizations of collision risk help users understand potential damages.

Abstract

Placing objects is a fundamental task for domestic service robots (DSRs). Thus, inferring the collision-risk before a placing motion is crucial for achieving the requested task. This problem is particularly challenging because it is necessary to predict what happens if an object is placed in a cluttered designated area. We show that a rule-based approach that uses plane detection, to detect free areas, performs poorly. To address this, we develop PonNet, which has multimodal attention branches and a self-attention mechanism to predict damaging collisions, based on RGBD images. Our method can visualize the risk of damaging collisions, which is convenient because it enables the user to understand the risk. For this purpose, we build and publish an original dataset that contains 12,000 photo-realistic images of specific placing areas, with daily life objects, in home environments. The experimental results show that our approach improves accuracy compared with the baseline methods.