An End-to-End Robot Architecture to Manipulate Non-Physical State Changes of Objects

TL;DR

This paper presents an integrated end-to-end robot architecture enabling humanoid robots to manipulate non-physical state changes in objects, demonstrated by playing the 2048 game with high accuracy.

Contribution

The paper introduces a novel architecture connecting sensors to actuators for complex non-physical state manipulation tasks in humanoid robots.

Findings

Successfully played 2048 game with high accuracy

Demonstrated architecture's effectiveness in real-world robot

Achieved performance comparable to simulation

Abstract

With the advance in robotic hardware and intelligent software, humanoid robot plays an important role in various tasks including service for human assistance and heavy job for hazardous industry. Recent advances in task learning enable humanoid robots to conduct dexterous manipulation tasks such as grasping objects and assembling parts of furniture. Operating objects without physical movements is an even more challenging task for humanoid robot because effects of actions may not be clearly seen in the physical configuration space and meaningful actions could be very complex in a long time horizon. As an example, playing a mobile game in a smart device has such challenges because both swipe actions and complex state transitions inside the smart devices in a long time horizon. In this paper, we solve this problem by introducing an integrated architecture which connects end-to-end dataflow from sensors to actuators in a humanoid robot to operate smart devices. We implement our integrated architecture in the Baxter Research Robot and experimentally demonstrate that the robot with our architecture could play a challenging mobile game, the 2048 game, as accurate as in a simulated environment.