An Empirical Study and Analysis of Learning Generalizable Manipulation Skill in the SAPIEN Simulator

TL;DR

This paper presents an empirical study on learning generalizable manipulation skills in the SAPIEN simulator, utilizing a deep transformer network to predict robot actions based on point cloud features, with promising leaderboard results.

Contribution

It introduces an end-to-end pipeline combining point cloud feature extraction and transformer-based action prediction, along with an empirical analysis of techniques for manipulation learning.

Findings

Achieved a promising ranking in the SAPIEN ManiSkill Challenge 2021

Developed an end-to-end manipulation skill learning pipeline

Provided insights through an empirical study with various tricks

Abstract

This paper provides a brief overview of our submission to the no interaction track of SAPIEN ManiSkill Challenge 2021. Our approach follows an end-to-end pipeline which mainly consists of two steps: we first extract the point cloud features of multiple objects; then we adopt these features to predict the action score of the robot simulators through a deep and wide transformer-based network. More specially, %to give guidance for future work, to open up avenues for exploitation of learning manipulation skill, we present an empirical study that includes a bag of tricks and abortive attempts. Finally, our method achieves a promising ranking on the leaderboard. All code of our solution is available at https://github.com/liu666666/bigfish\_codes.