SA-Net: Deep Neural Network for Robot Trajectory Recognition from RGB-D Streams

TL;DR

SA-Net is a deep neural network designed to recognize robot state-action pairs from RGB-D streams, improving accuracy and generalizing across different robotic applications for enhanced learning from demonstration.

Contribution

The paper introduces SA-Net, a novel deep neural network architecture that effectively recognizes state-action pairs from RGB-D data in robotic learning tasks, outperforming traditional image processing methods.

Findings

SA-Net achieves higher accuracy than traditional methods.

It generalizes well across different robotic platforms.

Physical robot deployment confirms improved recognition performance.

Abstract

Learning from demonstration (LfD) and imitation learning offer new paradigms for transferring task behavior to robots. A class of methods that enable such online learning require the robot to observe the task being performed and decompose the sensed streaming data into sequences of state-action pairs, which are then input to the methods. Thus, recognizing the state-action pairs correctly and quickly in sensed data is a crucial prerequisite for these methods. We present SA-Net a deep neural network architecture that recognizes state-action pairs from RGB-D data streams. SA-Net performed well in two diverse robotic applications of LfD -- one involving mobile ground robots and another involving a robotic manipulator -- which demonstrates that the architecture generalizes well to differing contexts. Comprehensive evaluations including deployment on a physical robot show that \sanet{} significantly improves on the accuracy of the previous method that utilizes traditional image processing and segmentation.