Simulated Mental Imagery for Robotic Task Planning

TL;DR

This paper introduces SiMIP, a novel sub-symbolic planning method for robots that uses simulated mental imagery with neural networks to generate and evaluate action plans without symbolic domain descriptions.

Contribution

It presents a new approach combining perception, simulated action, and success-checking in mental imagery for robotic planning, avoiding symbolic encoding.

Findings

Successfully implemented mental imagery-based planning in robots.

Achieved effective packing task performance with high success rates.

Demonstrated the method's efficiency on real scene datasets.

Abstract

Traditional AI-planning methods for task planning in robotics require a symbolically encoded domain description. While powerful in well-defined scenarios, as well as human-interpretable, setting this up requires substantial effort. Different from this, most everyday planning tasks are solved by humans intuitively, using mental imagery of the different planning steps. Here we suggest that the same approach can be used for robots, too, in cases which require only limited execution accuracy. In the current study, we propose a novel sub-symbolic method called Simulated Mental Imagery for Planning (SiMIP), which consists of perception, simulated action, success-checking and re-planning performed on 'imagined' images. We show that it is possible to implement mental imagery-based planning in an algorithmically sound way by combining regular convolutional neural networks and generative adversarial networks. With this method, the robot acquires the capability to use the initially existing scene to generate action plans without symbolic domain descriptions, while at the same time plans remain human-interpretable, different from deep reinforcement learning, which is an alternative sub-symbolic approach. We create a dataset from real scenes for a packing problem of having to correctly place different objects into different target slots. This way efficiency and success rate of this algorithm could be quantified.