Constraining Gaussian Process Implicit Surfaces for Robot Manipulation via Dataset Refinement

TL;DR

This paper introduces COGIS, an online learning method that refines Gaussian process implicit surfaces to improve robot manipulation in partially observable environments, enabling better obstacle avoidance and task success.

Contribution

The paper presents a novel dataset refinement technique for Gaussian process implicit surfaces that enhances obstacle modeling for model predictive control in manipulation tasks.

Findings

Successfully applied to peg-in-hole and deformable object tasks

Achieved 10/10 success rate on real-world cable manipulation

Outperformed baseline in partial observability scenarios

Abstract

Model-based control faces fundamental challenges in partially-observable environments due to unmodeled obstacles. We propose an online learning and optimization method to identify and avoid unobserved obstacles online. Our method, Constraint Obeying Gaussian Implicit Surfaces (COGIS), infers contact data using a combination of visual input and state tracking, informed by predictions from a nominal dynamics model. We then fit a Gaussian process implicit surface (GPIS) to these data and refine the dataset through a novel method of enforcing constraints on the estimated surface. This allows us to design a Model Predictive Control (MPC) method that leverages the obstacle estimate to complete multiple manipulation tasks. By modeling the environment instead of attempting to directly adapt the dynamics, our method succeeds at both low-dimensional peg-in-hole tasks and high-dimensional deformable object manipulation tasks. Our method succeeds in 10/10 trials vs 1/10 for a baseline on a real-world cable manipulation task under partial observability of the environment.