Morphology-Agnostic Visual Robotic Control

TL;DR

MAVRIC is a novel visual control method that enables robots to self-recognize and control their movements using minimal prior knowledge, suitable for various morphologies and unsteady camera setups.

Contribution

Introduces MAVRIC, a morphology-agnostic visual control approach that uses mutual information for self-recognition and control point discovery in real-time.

Findings

Works with unknown robot morphologies and camera poses

Effective in control tasks like 3D point reaching and trajectory following

Handles imprecise actuation and unsteady handheld cameras

Abstract

Existing approaches for visuomotor robotic control typically require characterizing the robot in advance by calibrating the camera or performing system identification. We propose MAVRIC, an approach that works with minimal prior knowledge of the robot's morphology, and requires only a camera view containing the robot and its environment and an unknown control interface. MAVRIC revolves around a mutual information-based method for self-recognition, which discovers visual "control points" on the robot body within a few seconds of exploratory interaction, and these control points in turn are then used for visual servoing. MAVRIC can control robots with imprecise actuation, no proprioceptive feedback, unknown morphologies including novel tools, unknown camera poses, and even unsteady handheld cameras. We demonstrate our method on visually-guided 3D point reaching, trajectory following, and robot-to-robot imitation.