Semi-Supervised Neural Processes for Articulated Object Interactions

TL;DR

This paper presents Semi-Supervised Neural Processes (SSNP), a model that leverages abundant passive visual data and limited labeled interactions to improve reward prediction in robotic object manipulation, reducing data and training requirements.

Contribution

The introduction of SSNP, a semi-supervised model that jointly trains on labeled and passive data for adaptive reward prediction in robotic tasks.

Findings

SSNP outperforms other semi-supervised methods in door-opening tasks.

SSNP requires less labeled data and training time.

SSNP effectively generalizes from passive visual data.

Abstract

The scarcity of labeled action data poses a considerable challenge for developing machine learning algorithms for robotic object manipulation. It is expensive and often infeasible for a robot to interact with many objects. Conversely, visual data of objects, without interaction, is abundantly available and can be leveraged for pretraining and feature extraction. However, current methods that rely on image data for pretraining do not easily adapt to task-specific predictions, since the learned features are not guaranteed to be relevant. This paper introduces the Semi-Supervised Neural Process (SSNP): an adaptive reward-prediction model designed for scenarios in which only a small subset of objects have labeled interaction data. In addition to predicting reward labels, the latent-space of the SSNP is jointly trained with an autoencoding objective using passive data from a much larger set of objects. Jointly training with both types of data allows the model to focus more effectively on generalizable features and minimizes the need for extensive retraining, thereby reducing computational demands. The efficacy of SSNP is demonstrated through a door-opening task, leading to better performance than other semi-supervised methods, and only using a fraction of the data compared to other adaptive models.