Put the Bear on the Chair! Intelligent Robot Interaction with Previously Unseen Chairs via Robot Imagination

TL;DR

This paper introduces a novel robotic system that enables autonomous seating of a teddy bear on previously unseen chairs by imagining sitting poses through virtual simulation, incorporating motion planning and human assistance for inaccessible chairs.

Contribution

The paper presents a new method combining virtual imagination, motion planning, and human assistance to enable robots to seat objects on unseen chairs in diverse scenarios.

Findings

High success rate in seating teddy bears on 12 unseen chairs.

Effective human assistance module for inaccessible chairs.

Robust system tested in various accessible and inaccessible poses.

Abstract

In this paper, we study the problem of autonomously seating a teddy bear on a previously unseen chair. To achieve this goal, we present a novel method for robots to imagine the sitting pose of the bear by physically simulating a virtual humanoid agent sitting on the chair. We also develop a robotic system which leverages motion planning to plan SE(2) motions for a humanoid robot to walk to the chair and whole-body motions to put the bear on it. Furthermore, to cope with cases where the chair is not in an accessible pose for placing the bear, a human assistance module is introduced for a human to follow language instructions given by the robot to rotate the chair and help make the chair accessible. We implement our method with a robot arm and a humanoid robot. We calibrate the proposed system with 3 chairs and test on 12 previously unseen chairs in both accessible and inaccessible poses extensively. Results show that our method enables the robot to autonomously seat the teddy bear on the 12 previously unseen chairs with a very high success rate. The human assistance module is also shown to be very effective in changing the accessibility of the chair. Video demos and more details are available at https://chirikjianlab.github.io/putbearonchair/.