Autonomously Learning to Visually Detect Where Manipulation Will Succeed

TL;DR

This paper presents a method for a robot to autonomously learn to predict successful manipulation locations using visual features and active learning, enabling it to improve its success rate in tasks like flipping switches and opening drawers.

Contribution

The paper introduces an autonomous learning framework that trains SVM classifiers for manipulation success prediction using visual features and active learning in a simulated home environment.

Findings

Robot achieved high success rates in manipulation tasks

Active learning improved training efficiency

The approach adapts to failures over time

Abstract

Visual features can help predict if a manipulation behavior will succeed at a given location. For example, the success of a behavior that flips light switches depends on the location of the switch. Within this paper, we present methods that enable a mobile manipulator to autonomously learn a function that takes an RGB image and a registered 3D point cloud as input and returns a 3D location at which a manipulation behavior is likely to succeed. Given a pair of manipulation behaviors that can change the state of the world between two sets (e.g., light switch up and light switch down), classifiers that detect when each behavior has been successful, and an initial hint as to where one of the behaviors will be successful, the robot autonomously trains a pair of support vector machine (SVM) classifiers by trying out the behaviors at locations in the world and observing the results. When an image feature vector associated with a 3D location is provided as input to one of the SVMs, the SVM predicts if the associated manipulation behavior will be successful at the 3D location. To evaluate our approach, we performed experiments with a PR2 robot from Willow Garage in a simulated home using behaviors that flip a light switch, push a rocker-type light switch, and operate a drawer. By using active learning, the robot efficiently learned SVMs that enabled it to consistently succeed at these tasks. After training, the robot also continued to learn in order to adapt in the event of failure.